Coding pattern comprising columns and rows of coordinate data

ABSTRACT

A substrate having a coding pattern disposed on a surface thereof. The coding pattern comprises a plurality of contiguous square tags of length l, each tag comprising x-coordinate data and y-coordinate data. A plurality of data elements are contained in each tag. The x-coordinate data is represented by a respective set of data elements and the y-coordinate data is represented by a respective set of data elements. All the x-coordinate data is represented in a column of the tag parallel with a y-axis, and all the y-coordinate data is represented in a row of the tag parallel with an x-axis. The column and the row each have a width v, such that any square portion of the coding pattern having a length (l+v) is guaranteed to contain the x-coordinate data and the y-coordinate data for a tag irrespective of whether a whole tag is contained in that portion.

COPENDING APPLICATIONS

The following applications have been filed by the Applicantsimultaneously with the present application:

-   -   NPT100US NPT102US NPT103US

The disclosures of these co-pending applications are incorporated hereinby reference. The above applications have been identified by theirfiling docket number, which will be substituted with the correspondingapplication number, once assigned.

CROSS REFERENCES

The following patents or patent applications filed by the applicant orassignee of the present invention are hereby incorporated bycross-reference.

10/815,621 10/815,635 10/815,647 11/488,162 10/815,636 11/041,65211/041,609 11/041,556 10/815,609 7,204,941 7,278,727 10/913,3807,122,076 7,156,289 09/575,197 6,720,985 7,295,839 09/722,174 7,068,3827,094,910 7,062,651 6,644,642 6,549,935 6,987,573 6,727,996 6,760,1197,064,851 6,290,349 6,428,155 6,785,016 6,831,682 6,741,871 6,965,43910/932,044 6,870,966 6,474,888 6,724,374 6,788,982 7,263,270 6,788,2936,737,591 09/693,514 10/778,056 10/778,061 11/193,482 7,055,7396,830,196 7,182,247 7,082,562 10/409,864 7,108,192 10/492,169 10/492,15210/492,168 10/492,161 7,308,148 6,957,768 7,170,499 11/856,06111/672,522 11/672,950 11/754,310 12/015,507 7,148,345 12/025,74612/025,762 12/025,765 10/407,212 6,902,255 6,755,509 12/178,61112/178,619

FIELD OF INVENTION

The present invention relates to a position-coding pattern on a surface.

BACKGROUND

The Applicant has previously described a method of enabling users toaccess information from a computer system via a printed substrate e.g.paper. The substrate has a coding pattern printed thereon, which is readby an optical sensing device when the user interacts with the substrateusing the sensing device. A computer receives interaction data from thesensing device and uses this data to determine what action is beingrequested by the user. For example, a user may make handwritten inputonto a form or make a selection gesture around a printed item. Thisinput is interpreted by the computer system with reference to a pagedescription corresponding to the printed substrate.

It would desirable to improve the coding pattern printed on thesubstrate so as to maximize usage of images captured by the sensingdevice. It would be further desirable to provide variants of a positioncoding pattern, suitable for printing by different types of printer,where each variant is readable by the same optical reader.

SUMMARY OF INVENTION

In a first aspect, the present invention provides a substrate having afirst coding pattern disposed on a surface thereof, the first codingpattern comprising:

-   -   a plurality of target elements defining a target grid, the        target grid comprising a plurality of cells, wherein neighboring        cells share target elements;    -   a plurality of data elements contained in each cell; and    -   a plurality of tags, each tag being defined by a first set of        contiguous cells, each tag comprising respective tag data        encoded by a respective set of the data elements, wherein each        cell comprises one or more registration symbols encoded by a        respective set of the data elements, each of the one or more        registration symbols identifying the cell as being contained in        the first set and thereby contained in the first coding pattern.

The registration symbols advantageously provide a means by which thefirst coding pattern can be distinguished from other coding pattern(s)of the same general type.

Optionally, a number of cells contained in the first set identifies thefirst set and thereby the first coding pattern. Optionally, each of theregistration symbols identifies the number of cells contained in thefirst set. Optionally, each registration symbol distinguishes the firstcoding pattern from a second coding pattern. Advantageously, theregistration symbols identify the first coding pattern by identifyingthe number of cells contained in each tag. For example, a first codingpattern have contain nine cells per tag, whilst a second coding patternmay have four cells per tag.

Optionally, the first coding pattern and the second coding pattern areboth readable and decodable by a same optical reader. Hence, the Netpagepen can read and decode different Netpage position-coding patterns,irrespective of the actual number of cells per tag. This enablesdifferent position-coding patterns to be used and printed, depending onthe print capabilities of a printer

Optionally, the second coding pattern comprises:

-   -   a plurality of target elements defining a target grid, the        target grid comprising a plurality of cells, wherein neighboring        cells share target elements;    -   a plurality of data elements contained in each cell; and    -   a plurality of tags, each tag being defined by a second set of        contiguous cells, each tag comprising respective tag data        encoded by a respective set of the data elements, wherein the        second set contains a different number of cells than the first        set.

Hence, the second-position coding pattern is typically of the samegeneral type as the first position-coding pattern.

Optionally, each cell of the first coding pattern and the second codingpattern comprises one or more registration symbols. A large number ofregistration symbols in each tag provides a high degree of redundancy,meaning that the pen can robustly recognize a particular coding pattern.

Optionally, the registration symbols in the first and second codingpatterns are configured and positioned identically relative to targetelements contained by each cell. The relative positioning of theregistration symbols in both the first and second coding patterns is thesame, so that the pen can find the registrations symbols before itidentifies which coding pattern it is reading.

Optionally, the second coding pattern is adapted to be printed at alower print resolution than the first coding pattern. Optionally, thesecond set contains a fewer number of cells than the first set.

Optionally, each registration symbol identifies a translation of thecell relative to a tag containing the cell. This enables alignment ofthe tag(s) with the target grid. Typically, a first translation codeword(e.g. 0, 1, 2) is reserved for the first coding pattern, whilst a secondtranslation codeword (e.g. 3, 4) is reserved for the second codingpattern.

Optionally, each cell comprises a pair of orthogonal registrationsymbols, each orthogonal registration symbol identifying a respectiveorthogonal translation of the cell relative to a tag containing thecell. Hence, each registration symbol is identifies either anx-translation or a y-translation of a cell relative to a tag containingthat cell.

Optionally, each tag is square and comprises M² contiguous square cells,wherein M is an integer having a value of at least 2. Typical tag sizesare M=2, 3 or 4. Preferably, M=2 or 3.

Optionally, M registration symbols in a row of M cells define a cyclicposition code having minimum distance M, the code being defined by afirst translation codeword.

Optionally, M registration symbols in a column of M cells define acyclic position code having minimum distance M, the code being definedby a second translation codeword.

Optionally, each tag comprises N cells, and at least N registrationsymbols form a third translation codeword with minimum distance N,wherein N is an integer having a value of at least 4.

Advantageously, the first, second and/or third translation codewordsenable robust alignment of the tag(s) with the target. For example, in atag containing nine cells, four symbol errors in the third translationcodeword may be corrected.

Optionally, each registration symbol further identifies an orientationof a layout of the tag data with respect to the target grid. The encodedorientation enables the Netpage pen to determine its orientation (yaw)relative to the tag data, and hence relative to the substrate.

Typically, the data elements are macrodots (i.e. readable marks in theform of dots). Typically, a portion of data is represented by mmacrodots, each of the macrodots occupying a respective position from aplurality of predetermined possible positions p within the cell, therespective positions of the macrodots representing one of a plurality ofpossible data values, wherein m is an integer of 1 or more (usually 2 ormore), and p>m (typically p≧2m). Encoding by multi-PPM in this wayensures uniform coverage of the substrate with macrodots, which helps toreduce visibility. Moreover, PPM encoding provides an internalluminescence reference for reading macrodots. For example, the darkest mmacrodots in the p positions are taken to be the PPM data, without theneed to refer to any external luminescence threshold value.

Optionally, each cell defines a symbol group, each symbol groupcomprising a plurality of Reed-Solomon symbols encoded by a plurality ofthe data elements.

Optionally, at least some of said tag data is encoded as a localcodeword comprised of a set of the Reed-Solomon symbols. The local tagdata typically identifies a location of the tag.

In a second aspect, there is provided a method of imaging either a firstcoding pattern or a second coding pattern disposed on a surface, themethod comprising the steps of:

(a) operatively positioning an optical reader relative to the surfacehaving either the first or second coding pattern disposed thereon;

(b) capturing an image of a portion of the first or second codingpattern, the first and second coding patterns each comprising:

-   -   a plurality of target elements defining a target grid, the        target grid comprising a plurality of cells, wherein neighboring        cells share target elements;    -   a plurality of data elements contained in each cell; and    -   a plurality of tags, each tag being defined by a set of        contiguous cells, each tag comprising respective tag data        encoded by a respective set of the data elements, wherein each        cell comprises one or more registration symbols encoded by a        respective set of the data elements;

(c) sampling and decoding at least one registration symbol contained inthe imaged portion;

(d) determining, from the decoded registration symbol, an identifier ofthe first or second coding pattern, the identifier indicating a numberof cells contained in each tag;

(e) determining, from the identifier, whether the optical reader ispositioned relative to the first coding pattern or the second codingpattern; and

(f) using the indicated number of cells to sample and decode the tagdata, wherein a number of cells contained in each tag of the firstcoding pattern is different from a number of cells contained in each tagof the second coding pattern.

Optionally, each registration symbol identifies a translation of thecell relative to a tag containing the cell, the method furthercomprising the step of:

using the translation to sample and decode the tag data.

Optionally, each registration symbol identifies an orientation of alayout of the tag data with respect to the target grid, the methodfurther comprising the step of:

using the orientation to sample and decode the tag data.

In a third aspect, there is provided a system for imaging either a firstcoding pattern or a second coding pattern, the system comprising:

-   (A) a substrate having either the first coding pattern or the second    coding pattern disposed on a surface thereof, wherein the first and    second coding patterns each comprises:

a plurality of target elements defining a target grid, the target gridcomprising a plurality of cells, wherein neighboring cells share targetelements;

a plurality of data elements contained in each cell; and

a plurality of tags, each tag being defined by a set of contiguouscells, each tag comprising respective tag data encoded by a respectiveset of the data elements, wherein each cell comprises one or moreregistration symbols encoded by a respective set of the data elements;

-   (B) an optical reader comprising:

an image sensor for capturing an image of a portion of the first orsecond coding pattern; and

a processor configured for performing the steps of:

-   -   (i) sampling and decoding at least one registration symbol        contained in the imaged portion;    -   (ii) determining, from the decoded registration symbol, an        identifier of the first or second coding pattern, the identifier        indicating a number of cells contained in each tag;    -   (iii) determining, from the identifier, whether the optical        reader is positioned relative to the first coding pattern or the        second coding pattern; and    -   (iv) using the indicated number of cells to sample and decode        the tag data, wherein a number of cells contained in each tag of        the first coding pattern is different from a number of cells        contained in each tag of the second coding pattern.

Optionally, each registration symbol identifies a translation of thecell relative to a tag containing the cell, the processor beingconfigured to perform the further step of:

using the translation to sample and decode the tag data.

Optionally, each registration symbol identifies an orientation of alayout of the tag data with respect to the target grid, the processorbeing configure to perform the further step of:

using the orientation to sample and decode the tag data.

In a fourth aspect, there is provided an optical reader for imagingeither a first coding pattern or a second coding pattern, the first andsecond coding patterns each comprising:

a plurality of target elements defining a target grid, the target gridcomprising a plurality of cells, wherein neighboring cells share targetelements;

a plurality of data elements contained in each cell; and

a plurality of tags, each tag being defined by a set of contiguouscells, each tag comprising respective tag data encoded by a respectiveset of the data elements, wherein each cell comprises one or moreregistration symbols encoded by a respective set of the data elements;

-   the optical reader comprising:

an image sensor for capturing an image of a portion of either first orsecond coding pattern; and

a processor configured for performing the steps of:

-   -   (i) sampling and decoding at least one registration symbol        contained in the imaged portion;    -   (ii) determining, from the decoded registration symbol, an        identifier of the first or second coding pattern, the identifier        indicating a number of cells contained in each tag;    -   (iii) determining, from the identifier, whether the optical        reader is positioned relative to the first coding pattern or the        second coding pattern; and    -   (iv) using the indicated number of cells to sample and decode        the tag data, wherein a number of cells contained in each tag of        the first coding pattern is different from a number of cells        contained in each tag of the second coding pattern.

It will be appreciated that optional embodiments of the first aspect mayalso be optional embodiments of the second, third or fourth aspects.

In a fifth aspect, there is provided a substrate having a coding patterndisposed on a surface thereof, the coding pattern comprising:

a plurality of contiguous square tags of length l, each tag comprisingx-coordinate data and y-coordinate data; and

a plurality of data elements contained in each tag, the x-coordinatedata being represented by a respective set of data elements and they-coordinate data being represented by a respective set of dataelements, wherein:

all the x-coordinate data is represented in a column of the tag parallelwith a y-axis;

all the y-coordinate data is represented in a row of the tag parallelwith an x-axis; and

the column and the row each have a width v, such that any square portionof the coding pattern having a length (l+v) is guaranteed to contain thex-coordinate data and the y-coordinate data for a tag irrespective ofwhether a whole tag is contained in the portion.

The fifth aspect of the invention advantageously enables non-replicationof coordinate data in each tag, which saves on tag space. Typically,x-coordinate data should be replicated in each vertical half of tag, andy-coordinate data should be replicated in each horizontal half of a tag.However, by encoding all x-coordinate data in one column, and ally-coordinate data in one row, the requirement for replication isobviated. If the column or row has a width v, then any square portion oflength (l+v) is guaranteed to contain the relevant coordinate data.Moreover, if the width v corresponds to a width or length of acoordinate data symbol (depending on the shape and orientation ofcoordinate data symbols in the column or row), then it is ensured thatany square portion of length (l+v) is guaranteed to contain the relevantcoordinate data from spatially coherent samples i.e. from the samesymbol, as opposed to partial symbols at opposite sides of a field ofview.

Optionally, a plurality of target elements define a target grid, thetarget grid comprising a plurality of cells, wherein neighboring cellsshare target elements and wherein each tag is defined by a plurality ofcontiguous cells.

Optionally, each tag comprises M² contiguous square cells, wherein M isan integer having a value of at least 1. Typical tag sizes are M=2, 3 or4. Preferably, M=2 or 3

Optionally, the data elements are macrodots, which are readable dot-likemarks formed by a plurality of contiguous printed dots.

Optionally, v=ts, wherein: s is defined as a spacing between adjacentmacrodots; and t is an integer value of 2 or more.

Optionally, the macrodots encode data values by pulse positionmodulation (PPM).

Optionally, a portion of data is represented by m macrodots, each of themacrodots occupying a respective position from a plurality ofpredetermined possible positions p within the cell, the respectivepositions of the macrodots representing one of a plurality of possibledata values.

Optionally, the x-coordinate data is encoded as an x-coordinate codewordcomprised of a respective set of the X-Reed-Solomon symbols, and they-coordinate data is encoded as a y-coordinate codeword comprised of arespective set of the Y-Reed-Solomon symbols.

Optionally, the X-Reed-Solomon symbols are configured and oriented inthe column so as to have the width v, and wherein the Y-Reed-Solomonsymbols are configured and oriented in the row so as to have the widthv.

Optionally, each tag comprises a plurality of common codewords, eachcommon codeword being comprised of a respective set of the Reed-Solomonsymbols, wherein the plurality of common codewords are defined ascodewords common to a plurality of contiguous tags. A common codewordtypically encodes a region ID or page ID for the substrate.

Optionally, each symbol group comprises a fragment of at least one ofthe common codewords, and contiguous symbol groups are arranged suchthat any tag-sized portion of the coding pattern is guaranteed tocontain the plurality of common codewords irrespective of whether awhole tag is contained in the portion.

Optionally, each cell comprises a registration symbol encoded by arespective set of the data elements, the registration symbol identifyingone or more of:

a translation of the cell relative to a tag containing the cell;

an orientation of a layout of tag data with respect to the target grid;

a number of cells in each tag;

a flag associated with the tag.

Optionally, each cell comprises first and second registration symbols,the first registration symbol identifying a first orthogonal translationof the cell, the second registration symbol identifying a secondorthogonal translation of the cell.

Optionally, the first registration symbol identifies a first directioncomponent of the orientation, and the second registration symbolidentifies a second direction component of the orientation, such thatthe first and second orthogonal registration symbols together identifythe orientation via the first and second direction components.

Optionally, the target elements are target dots and the data elementsare macrodots, and each target dot has a diameter of at least twice thatof each macrodot. This enables low-pass filtration of captured images toretain target elements but obscure macrodots.

In a sixth aspect, there is provided a method of imaging a codingpattern disposed on a surface of a substrate, the method comprising thesteps of:

(a) operatively positioning an optical reader relative to the surfaceand capturing an image of a portion of the coding pattern, the codingpattern comprising:

a plurality of contiguous square tags of length l, each tag comprisingx-coordinate data and y-coordinate data; and

a plurality of data elements contained in each tag, the x-coordinatedata being represented by a respective set of data elements and they-coordinate data being represented by a respective set of dataelements, wherein:

all the x-coordinate data is represented in a column of the tag parallelwith a y-axis;

all the y-coordinate data is represented in a row of the tag parallelwith an x-axis; and

the column and the row each have a width v,

(b) sampling and decoding x-coordinate data and y-coordinate data withinthe imaged portion; and

(c) determining a position of the reader, wherein the imaged portion hasa diameter of at least (l+v)√2 and less than (2l)√2.

Since the field of view of the optical reader is not required to have adiameter of at least two tag diameters, then the imaging requirements ofthe reader are reduced. Hence, the position-coding pattern not onlyprovides efficient use of tag space, but also allows the imaging fieldof view of the tag reader to be minimized.

Optionally, each tag comprises a plurality of common codewords, eachcommon codeword being comprised of a respective set of the Reed-Solomonsymbols, wherein the plurality of common codewords are defined ascodewords common to a plurality of contiguous tags, the method furthercomprising the step of:

sampling and decoding the common codeword within the imaged portion.

Optionally, one or more of the common codewords encode region identitydata uniquely identifying a region of the surface, the method furthercomprising:

determining the an identity of the region.

Optionally, the region identity data uniquely identifies the substrate.

In a seventh aspect, there is provided a system for imaging a codingpattern disposed on a surface of a substrate, the system comprising:

-   (A) the substrate, wherein the coding pattern comprises:

a plurality of contiguous square tags of length l, each tag comprisingx-coordinate data and y-coordinate data; and

a plurality of data elements contained in each tag, the x-coordinatedata being represented by a respective set of data elements and they-coordinate data being represented by a respective set of dataelements, wherein:

all the x-coordinate data is represented in a column of the tag parallelwith a y-axis;

all the y-coordinate data is represented in a row of the tag parallelwith an x-axis; and

the column and the row each have a width v,

-   (B) an optical reader comprising:

an image sensor for capturing an image of a portion of the codingpattern, the image sensor having a field-of-view of at least (l+v)√2 andless than (2l)√2; and

a processor configured for performing the steps of:

-   -   (i) sampling and decoding x-coordinate data and y-coordinate        data contained in an imaged portion; and    -   (ii) determining a position of the reader.

In an eighth aspect, there is provided an optical reader for imaging acoding pattern disposed on a surface of a substrate, the coding patterncomprising:

a plurality of contiguous square tags of length l, each tag comprisingx-coordinate data and y-coordinate data; and

a plurality of data elements contained in each tag, the x-coordinatedata being represented by a respective set of data elements and they-coordinate data being represented by a respective set of dataelements, wherein:

all the x-coordinate data is represented in a column of the tag parallelwith a y-axis;

all the y-coordinate data is represented in a row of the tag parallelwith an x-axis; and

the column and the row each have a width v,

-   the optical reader comprising:

an image sensor for capturing an image of a portion of the codingpattern, the image sensor having a field-of-view of at least (l+v)√2 andless than (2l)√2; and

a processor configured for performing the steps of:

-   -   (i) sampling and decoding x-coordinate data and y-coordinate        data contained in an imaged portion; and    -   (ii) determining a position of the reader.

It will be appreciated that optional embodiments of the fifth aspect mayalso be optional embodiments of sixth, seventh and eighth aspects.

BRIEF DESCRIPTION OF DRAWINGS

Preferred and other embodiments of the invention will now be described,by way of non-limiting example only, with reference to the accompanyingdrawings, in which:

FIG. 1 is a schematic of a relationship between a sample printed netpageand its online page description;

FIG. 2 shows an embodiment of basic netpage architecture with variousalternatives for the relay device;

FIG. 3 shows the structure of a tag for a first position-coding pattern;

FIG. 4 shows a group of twelve data symbols and four targets for thefirst position-coding pattern;

FIG. 5 shows the layout of a 2-6PPM and 3-6PPM data symbol for the firstposition-coding pattern;

FIG. 6 shows the spacing of macrodot positions in the firstposition-coding pattern;

FIG. 7 shows the layout of a 2-6PPM registration symbol for the firstposition-coding pattern;

FIG. 8 shows a semi-replicated x-coordinate codeword X for the firstposition-coding pattern;

FIG. 9 shows a semi-replicated y-coordinate codeword Y for the firstposition-coding pattern;

FIG. 10 shows common codewords A, B, C and D, with codeword A shown inbold outline for the first position-coding pattern;

FIG. 11 shows an optional codeword E for the first position-codingpattern;

FIG. 12 shows the layout of a complete tag for the first position-codingpattern;

FIG. 13 shows the layout of a Reed-Solomon codeword for the firstposition-coding pattern;

FIG. 14 shows the structure of a tag for a second position-codingpattern;

FIG. 15 shows a group of eight data symbols and four targets for thesecond position-coding pattern;

FIG. 16 shows the layout of a 2-9PPM data symbol for the secondposition-coding pattern;

FIG. 17 shows an x-coordinate codeword X for the second position-codingpattern;

FIG. 18 shows a y-coordinate codeword Y for the second position-codingpattern;

FIG. 19 shows a common codeword A for the second position-codingpattern;

FIG. 20 shows the layout of a complete tag for the secondposition-coding pattern;

FIG. 21 shows the layout of a Reed-Solomon codeword for the secondposition-coding pattern;

FIG. 22 is a flowchart of initial image processing by the Netpage pen;

FIG. 23 is a flowchart of codeword decoding subsequent to the initialimage processing;

FIG. 24 shows a nib and elevation of the Netpage pen held by a user;

FIG. 25 shows the pen held by a user at a typical incline to a writingsurface;

FIG. 26 is a lateral cross section through the pen;

FIG. 27A is a bottom and nib end partial perspective of the pen;

FIG. 27B is a bottom and nib end partial perspective with the fields ofillumination and field of view of the sensor window shown in dottedoutline;

FIG. 28 is a longitudinal cross section of the pen;

FIG. 29A is a partial longitudinal cross section of the nib and barrelmolding;

FIG. 29B is a partial longitudinal cross section of the IR LED's and thebarrel molding;

FIG. 30 is a ray trace of the pen optics adjacent a sketch of the inkcartridge;

FIG. 31 is a side elevation of the lens;

FIG. 32 is a side elevation of the nib and the field of view of theoptical sensor; and

FIG. 33 is a block diagram of the pen electronics.

DETAILED DESCRIPTION OF PREFERRED AND OTHER EMBODIMENTS 1.1 NetpageSystem Architecture

In a preferred embodiment, the invention is configured to work with thenetpage networked computer system, an overview of which follows. Inbrief summary, the preferred form of the netpage system employs acomputer interface in the form of a mapped surface, that is, a physicalsurface which contains references to a map of the surface maintained ina computer system. The map references can be queried by an appropriatesensing device. Depending upon the specific implementation, the mapreferences may be encoded visibly or invisibly, and defined in such away that a local query on the mapped surface yields an unambiguous mapreference both within the map and among different maps. The computersystem can contain information about features on the mapped surface, andsuch information can be retrieved based on map references supplied by asensing device used with the mapped surface. The information thusretrieved can take the form of actions which are initiated by thecomputer system on behalf of the operator in response to the operator'sinteraction with the surface features.

In its preferred form, the netpage system relies on the production of,and human interaction with, netpages. These are pages of text, graphicsand images printed on ordinary paper, but which work like interactivewebpages. Information is encoded on each page using ink which issubstantially invisible to the unaided human eye. The ink, however, andthereby the coded data, can be sensed by an optically imaging sensingdevice (or reader) and transmitted to the netpage system. The sensingdevice may take the form of a clicker (for clicking on a specificposition on a surface), a pointer having a stylus (for pointing orgesturing on a surface using pointer strokes), or a pen having a markingnib (for marking a surface with ink when pointing, gesturing or writingon the surface). References herein to “pen” or “netpage pen” areprovided by way of example only. It will, of course, be appreciated thatthe pen may take the form of any of the sensing devices or readersdescribed herein.

In one embodiment, active buttons and hyperlinks on each page can beclicked with the sensing device to request information from the networkor to signal preferences to a network server. In one embodiment, textwritten by hand on a netpage is automatically recognized and convertedto computer text in the netpage system, allowing forms to be filled in.In other embodiments, signatures recorded on a netpage are automaticallyverified, allowing e-commerce transactions to be securely authorized. Inother embodiments, text on a netpage may be clicked or gestured toinitiate a search based on keywords indicated by the user.

As illustrated in FIG. 1, a printed netpage 1 can represent ainteractive form which can be filled in by the user both physically, onthe printed page, and “electronically”, via communication between thepen and the netpage system. The example shows a “Request” formcontaining name and address fields and a submit button. The netpage 1consists of graphic data 2, printed using visible ink, and a surfacecoding pattern 3 superimposed with the graphic data. The surface codingpattern 3 comprises a collection of tags 4. A typical tag 4 is shown inthe shaded region of FIG. 1, although it will be appreciated thatcontiguous tags 4, defined by the coding pattern 3, are densely tiledover the whole netpage 1.

The corresponding page description 5, stored on the netpage network,describes the individual elements of the netpage. In particular itdescribes the type and spatial extent (zone) of each interactive element(i.e. text field or button in the example), to allow the netpage systemto correctly interpret input via the netpage. The submit button 6, forexample, has a zone 7 which corresponds to the spatial extent of thecorresponding graphic 8.

As illustrated in FIG. 2, a netpage sensing device 400, such as the pendescribed in Section 5, works in conjunction with a netpage relay device601, which is an Internet-connected device for home, office or mobileuse. The pen 400 is wireless and communicates securely with the netpagerelay device 601 via a short-range radio link 9. In an alternativeembodiment, the netpage pen 400 utilises a wired connection, such as aUSB or other serial connection, to the relay device 601.

The relay device 601 performs the basic function of relaying interactiondata to a page server 10, which interprets the interaction data. Asshown in FIG. 2, the relay device 601 may, for example, take the form ofa personal computer 601 a, a netpage printer 601 b or some other relay601 c (e.g. personal computer or mobile phone incorporating a webbrowser).

The netpage printer 601 b is able to deliver, periodically or on demand,personalized newspapers, magazines, catalogs, brochures and otherpublications, all printed at high quality as interactive netpages.Unlike a personal computer, the netpage printer is an appliance whichcan be, for example, wall-mounted adjacent to an area where the morningnews is first consumed, such as in a user's kitchen, near a breakfasttable, or near the household's point of departure for the day. It alsocomes in tabletop, desktop, portable and miniature versions. Netpagesprinted on-demand at their point of consumption combine the ease-of-useof paper with the timeliness and interactivity of an interactive medium.

Alternatively, the netpage relay device 601 may be a portable device,such as a mobile phone or PDA, a laptop or desktop computer, or aninformation appliance connected to a shared display, such as a TV. Ifthe relay device 601 is not a netpage printer 601 b which printsnetpages digitally and on demand, the netpages may be printed bytraditional analog printing presses, using such techniques as offsetlithography, flexography, screen printing, relief printing androtogravure, as well as by digital printing presses, using techniquessuch as drop-on-demand inkjet, continuous inkjet, dye transfer, andlaser printing.

As shown in FIG. 2, the netpage sensing device 400 interacts with aportion of the tag pattern on a printed netpage 1, or other printedsubstrate such as a label of a product item 251, and communicates, via ashort-range radio link 9, the interaction to the relay device 601. Therelay 601 sends corresponding interaction data to the relevant netpagepage server 10 for interpretation. Raw data received from the sensingdevice 400 may be relayed directly to the page server 10 as interactiondata. Alternatively, the interaction data may be encoded in the form ofan interaction URI and transmitted to the page server 10 via a user'sweb browser 601 c. The web browser 601 c may then receive a URI from thepage server 10 and access a webpage via a webserver 201. In somecircumstances, the page server 10 may access application computersoftware running on a netpage application server 13.

The netpage relay device 601 can be configured to support any number ofsensing devices, and a sensing device can work with any number ofnetpage relays. In the preferred implementation, each netpage sensingdevice 400 has a unique identifier. This allows each user to maintain adistinct profile with respect to a netpage page server 10 or applicationserver 13.

Digital, on-demand delivery of netpages 1 may be performed by thenetpage printer 601 b, which exploits the growing availability ofbroadband Internet access. Netpage publication servers 14 on the netpagenetwork are configured to deliver print-quality publications to netpageprinters. Periodical publications are delivered automatically tosubscribing netpage printers via pointcasting and multicasting Internetprotocols. Personalized publications are filtered and formattedaccording to individual user profiles.

A netpage pen may be registered with a netpage registration server 11and linked to one or more payment card accounts. This allows e-commercepayments to be securely authorized using the netpage pen. The netpageregistration server compares the signature captured by the netpage penwith a previously registered signature, allowing it to authenticate theuser's identity to an e-commerce server. Other biometrics can also beused to verify identity. One version of the netpage pen includesfingerprint scanning, verified in a similar way by the netpageregistration server.

1.2 Netpages

Netpages are the foundation on which a netpage network is built. Theyprovide a paper-based user interface to published information andinteractive services.

As shown in FIG. 1, a netpage consists of a printed page (or othersurface region) invisibly tagged with references to an onlinedescription 5 of the page. The online page description 5 is maintainedpersistently by the netpage page server 10. The page descriptiondescribes the visible layout and content of the page, including text,graphics and images. It also describes the input elements on the page,including buttons, hyperlinks, and input fields. A netpage allowsmarkings made with a netpage pen on its surface to be simultaneouslycaptured and processed by the netpage system.

Multiple netpages (for example, those printed by analog printingpresses) can share the same page description. However, to allow inputthrough otherwise identical pages to be distinguished, each netpage maybe assigned a unique page identifier. This page ID (or, more generally,region ID) has sufficient precision to distinguish between a very largenumber of netpages.

Each reference to the page description 5 is repeatedly encoded in thenetpage pattern. Each tag (and/or a collection of contiguous tags)identifies the unique page on which it appears, and thereby indirectlyidentifies the page description 5. Each tag also identifies its ownposition on the page. Characteristics of the tags are described in moredetail below.

Tags are typically printed in infrared-absorptive ink on any substratewhich is infrared-reflective, such as ordinary paper, or in infraredfluorescing ink. Near-infrared wavelengths are invisible to the humaneye but are easily sensed by a solid-state image sensor with anappropriate filter.

A tag is sensed by a 2D area image sensor in the netpage sensing device,and the tag data is transmitted to the netpage system via the nearestnetpage relay device 601. The pen 400 is wireless and communicates withthe netpage relay device 601 via a short-range radio link. It isimportant that the pen recognize the page ID and position on everyinteraction with the page, since the interaction is stateless. Tags areerror-correctably encoded to make them partially tolerant to surfacedamage.

The netpage page server 10 maintains a unique page instance for eachunique printed netpage, allowing it to maintain a distinct set ofuser-supplied values for input fields in the page description 5 for eachprinted netpage 1.

2 Netpage Tags 2.1 Tag Data Content

Each tag 4 identifies an absolute location of that tag within a regionof a substrate.

Each interaction with a netpage should also provide a region identitytogether with the tag location. In a preferred embodiment, the region towhich a tag refers coincides with an entire page, and the region ID istherefore synonymous with the page ID of the page on which the tagappears. In other embodiments, the region to which a tag refers can bean arbitrary subregion of a page or other surface. For example, it cancoincide with the zone of an interactive element, in which case theregion ID can directly identify the interactive element.

As described in the Applicant's previous applications (e.g. U.S. Pat.No. 6,832,717), the region identity may be encoded discretely in eachtag 4. As will be described in more detail below, the region identitymay be encoded by a plurality of contiguous tags in such a way thatevery interaction with the substrate still identifies the regionidentity, even if a whole tag is not in the field of view of the sensingdevice.

Each tag 4 should preferably identify an orientation of the tag relativeto the substrate on which the tag is printed. Orientation data read froma tag enables the rotation (yaw) of the pen 400 relative to thesubstrate to be determined

A tag 4 may also encode one or more flags which relate to the region asa whole or to an individual tag. One or more flag bits may, for example,signal a sensing device to provide feedback indicative of a functionassociated with the immediate area of the tag, without the sensingdevice having to refer to a description of the region. A netpage penmay, for example, illuminate an “active area” LED when in the zone of ahyperlink.

A tag 4 may also encode a digital signature or a fragment thereof. Tagsencoding (partial) digital signatures are useful in applications whereit is required to verify a product's authenticity. Such applications aredescribed in, for example, US Publication No. 2007/0108285, the contentsof which is herein incorporated by reference. The digital signature maybe encoded in such a way that it can be retrieved from every interactionwith the substrate. Alternatively, the digital signature may be encodedin such a way that it can be assembled from a random or partial scan ofthe substrate.

It will, of course, be appreciated that other types of information (e.g.tag size etc) may also be encoded into each tag or a plurality of tags,as will be explained in more detail below.

2.2 Position-Coding Pattern Variants

Although the adoption of a ubiquitous position-coding Netpage tagpattern for all users and all applications of the Netpage system isdesirable, there may be technological or other barriers to such aubiquitous coding pattern, at least during initial uptake of the Netpagesystem. One such barrier is a print resolution at which theposition-coding pattern is printed. The Netpage tag pattern isadvantageously designed to be printed using the Applicant'shigh-resolution (1600 dpi) pagewidth inkjet printers. The Netpage systemcomplements the Applicant's inkjet printers, which are able to printNetpages having a high degree of functionality and position resolutionvia the printed tags 4. Ideally, Netpages are printed using‘Netpage-aware’ printers, which are specifically tailored for printingNetpages.

However, the Netpage system is a generic page-based system that need notbe inextricably tied to such printers. Preferably, Netpages should beprintable using other types of printers, including existing lowerresolution (e.g. 300 dpi) print-on-demand printers, such as laserprinters and other inkjet printers. Netpages should also be printableusing traditional analogue printing presses, which use, for example,established offset, rotogravure or photogravure printing techniques.

Self-evidently, position-coding patterns designed to be printed withhigh-resolution printers may not be printable using lower resolutionprinting technologies. If a dot spacing in the coding pattern is toosmall, then individual dots may not be resolvable by a relatively lowresolution printing technology. Moreover, the resultant coding pattern,printed by a low-resolution printer, would not be readable by theNetpage pen 400 if adjacent dots are merged together.

One approach to this problem would be to provide a ubiquitous Netpagecoding pattern, which is suitable for printing by both low-resolutionand high-resolution printers. However, this is an unsatisfactorysolution to the problem, because the higher degree of functionality andresolution of Netpages printed by the Applicant's pagewidth inkjetprinters would be lost unnecessarily.

The present invention therefore provides two variants of the Netpageposition-coding pattern, both of which are readable by the same Netpagepen 400. A first position-coding pattern (dubbed “Yarrow” by the presentApplicant) is suitable for printing by the Applicant's high-resolution(1600 dpi) pagewidth inkjet printers and has a high degree offunctionality and resolution. A second position-coding pattern (dubbed“Saffron” by the present Applicant) is suitable for printing byrelatively low resolution (e.g. 300 dpi) printers and has a lower degreeof functionality and resolution.

Importantly, both the first and second position-coding patterns arereadable by the Netpage pen 400 by virtue of features common to eachpattern. The Netpage pen 400 is able to determine whether it is readingthe first or second position-coding pattern by decoding a registrationsymbol in each tag 4, as will be explained in more detail below. Oncethe pen 400 has recognized the coding pattern it is reading, decoding oftag data can proceed in accordance with that particular coding pattern.

3 First Position-Coding Pattern (“Yarrow”) 3.1 Background

An earlier version of the first position-coding pattern (“Yarrow”) wasdescribed in Applicant's U.S. application Ser. Nos. 12/178,611 and12/178,619 (Attorney Docket Nos. NPT087US and NPT092US). This earlierversion of the first position-coding pattern has been modified forcompatibility with the second position-coding pattern described hereinin Section 4. In particular, the registration symbols now map totranslation code symbol values specifically identifying the firstposition-coding pattern, as described in more detail in Section 3.6.1.The complete first position-coding pattern will now be described indetail below.

3.2 General Tag Structure

As described above in connection with FIG. 1, the netpage surface codinggenerally consists of a dense planar tiling of tags. In the firstposition-coding pattern (“Yarrow”), each tag 4 is represented by twokinds of elements. Referring to FIGS. 3 and 4, the first kind of elementis a target element. Target elements in the form of target dots 301allow a tag 4 to be located in an image of a coded surface, and allowthe perspective distortion of the tag to be inferred. The second kind ofelement is a data element in the form of a macrodot 302 (see FIG. 6).The macrodots 302 encode data values. As described in the Applicant'searlier disclosures (e.g. U.S. Pat. No. 6,832,717), the presence orabsence of a macrodot was be used to represent a binary bit. However,the tag structure of the first position-coding pattern encodes a datavalue using multi-pulse position modulation, which is described in moredetail in Section 3.3.

The coding pattern 3 is represented on the surface in such a way as toallow it to be acquired by an optical imaging system, and in particularby an optical system with a narrowband response in the near-infrared.The pattern 3 is typically printed onto the surface using a narrowbandnear-infrared ink.

FIG. 3 shows the structure of a complete tag 4A from the firstposition-coding pattern, with target elements 301 shown. The tag 4A issquare and contains sixteen target elements. Those target elements 301located at the edges and corners of the tag (twelve in total) are sharedby adjacent tags and define the perimeter of the tag. The high number oftarget elements 301 advantageously facilitates accurate determination ofa perspective distortion of the tag 4 when it is imaged by the sensingdevice 400. This improves the accuracy of tag sensing and, ultimately,position determination.

The tag 4A consists of a square array of nine symbol groups 303. Symbolgroups 303 are demarcated by the target elements 301 so that each symbolgroup is contained within a square defined by four target elements.Adjacent symbol groups 303 are contiguous and share targets.

Since the target elements 301 are all identical, they do not demarcateone tag from its adjacent tags. Viewed purely at the level of targetelements, only symbol groups 303, which define cells of a target grid,can be distinguished—the tags 4A themselves are indistinguishable byviewing only the target elements. Hence, tags 4A must be aligned withthe target grid as part of tag decoding.

The tag 4A is designed to allow all tag data, with the exception of anembedded data object (see Section 3.9.3), to be recovered from animaging field of view substantially the size of the tag.

3.3 Symbol Groups

As shown in FIG. 4, each of the nine symbol groups 303 comprises twelvedata symbols 304A, each data symbol being part of a codeword. Inaddition, each symbol group 303 comprises a pair of registrationsymbols—a vertical registration symbol (‘VRS’) and a horizontalregistration symbol (‘HRS’). These allow the orientation and/ortranslation of the tag 4A in the field of view to be determined.Translation refers to the translation of tag(s) relative to the symbolgroups 303 in the field of view. In other words, the registrationsymbols enable alignment of the ‘invisible’ tags with the target grid.

Each data symbol 304A is a multi-pulse position modulated (PPM) datasymbol. Typically, each PPM data symbol 304A encodes a single 4-bitReed-Solomon symbol using 3 macrodots in any of 6 positions {p₀, p₁, p₂,p₃, p₄, p₅}, i.e. using 3-6 pulse-position modulation (PPM). However, itwill be appreciated that other forms of multi-PPM encoding are equallypossible.

3-6PPM has a range of 20 codes, or 4.3 bits, and is used forReed-Solomon data symbols and Reed-Solomon redundancy symbols.

Each symbol group also contains a 2-6PPM vertical registration symbol(VRS) and a 2-6PPM horizontal registration symbol (HRS). These allow theorientation and translation of the tag in the field of view to bedetermined. This is described in more detail in Section 3.6.1.

FIG. 5 shows the layout for a 2-6PPM or 3-6PPM data symbol 304.

Table 1 defines the mapping from 3-6PPM symbol values to Reed-Solomonsymbol values. Unused symbol values can be treated as erasures.

TABLE 1 3-6PPM to Reed-Solomon symbol mapping Corresponding Reed-Solomon3-6PPM symbol symbol value value (p₅-p₀) (base 16) 000111 unused 001011unused 001101 0 001110 1 010011 2 010101 3 010110 4 011001 5 011010 6011100 7 100011 8 100101 9 100110 a 101001 b 101010 c 101100 d 110001 e110010 f 110100 unused 111000 unused

3.4 Targets and Macrodots

The spacing of macrodots 302 in both dimensions, as shown in FIG. 6, isspecified by the parameter s. It has a nominal value of 127 μm, based on8 dots printed at a pitch of 1600 dots per inch.

Only macrodots 302 are part of the representation of a symbol 304A inthe pattern. The outline of a symbol 304A is shown in, for example,FIGS. 3 and 4 merely to elucidate more clearly the structure of the tag4A.

A macrodot 302 is nominally square with a nominal size of (4/8)s.However, it is allowed to vary in size by ±10% according to thecapabilities of the device used to produce the pattern.

A target 301 is nominally circular with a nominal diameter of (12/8)s.However, it is allowed to vary in size by ±10% according to thecapabilities of the device used to produce the pattern.

Each symbol group 303 has a width of 10s. Therefore, each tag 4A has awidth of 30s and a length of 30s. However, it should be noted from FIG.3 that the tag 4A is configured so that some data symbols 304A extendbeyond the perimeter edge of the tag 4A by one macrodot unit (1s), andinterlock with complementary data symbols from adjacent tags. Thisarrangement provides a tessellated pattern of data symbols 304A withinthe target grid. From a data acquisition standpoint, tessellation ofdata symbols in this way increases the effective length of each tag 4Aby one macrodot unit.

The macrodot spacing, and therefore the overall scale of the tagpattern, is allowed to vary between 127 μm and 120 μm according to thecapabilities of the device used to produce the pattern. Any deviationfrom the nominal scale is recorded in each tag (via a macrodot size IDfield) to allow accurate generation of position samples.

These tolerances are independent of one another. They may be refinedwith reference to particular printer characteristics.

3.5 Field of View

As mentioned above, the tag 4A is designed to allow all tag data to berecovered from an imaging field of view roughly the size of the tag. Anydata common to a set of contiguous tags only needs to appear once withineach tag, since fragments of the common data can be recovered fromadjacent tags. Any data common only to a column or row of tags mayappear twice within the tag—i.e. once in each horizontal half orvertical half of the tag respectively. However, special symbolarrangements may be used to ameliorate this requirement, as described inmore detail in Section 3.6.3. Finally, any data unique to the tag mustappear four times within the tag—i.e. once in each quadrant.

Although data which is common to a set of tags, in one or both spatialdimensions, may be decoded from fragments from adjacent tags,pulse-position modulated values are best decoded from spatially-coherentsamples (i.e. from a whole symbol as opposed to partial symbols atopposite sides of the field of view), since this allows raw samplevalues to be compared without first being normalised. This implies thatthe field of view must be large enough to contain two complete copies ofeach such pulse-position modulated value. The tag is designed so thatthe maximum extent of a pulse-position modulated value is threemacrodots (see FIG. 3). Making the field of view at least as large asthe tag plus three macrodot units guarantees that pulse-positionmodulated values can be coherently sampled.

The only exceptions are the translation codes described in the nextsection, which are four macrodot units long. However, these are highlyredundant and the loss of up to four symbols at the edge of the field ofview is not a problem.

3.6 Encoded Codes and Codewords

In this section (Section 3.6), each symbol in FIGS. 8 to 12 is shownwith a unique label. The label consists of an alphabetic prefix whichidentifies which codeword the symbol is part of, and a numeric suffixwhich indicates the index of the symbol within the codeword. Forsimplicity only data symbols 304A are shown, not registration symbols.

Although some symbol labels are shown rotated to indicate the symmetryof the layout of certain codewords, the layout of each symbol isdetermined by its position within a symbol group and not by the rotationof the symbol label (as described in, for example, the Applicant's USPublication No. 2006/146069).

3.6.1 Registration Symbols

Each registration symbol is encoded using 2-6PPM. FIG. 7 shows thelayout of the registration symbol.

As shown in FIG. 4, the horizontal and vertical registration symbolseach appear once within a symbol group 303. The registration symbols ofan entire tag typically indicate the vertical and horizontal translationof the tag by coding two orthogonal translation codes, and theorientation of the tag by coding two orthogonal direction codes.

Each registration symbol may also encode a one-bit symbol of a flag code(see Section 3.6.2).

Table 2 defines the mapping from 2-6PPM registration symbol values toflag code, direction code and translation code symbol values.

TABLE 2 2-6PPM registration symbol values to flag code, direction codeand translation code symbol mapping 2-6PPM translation direction flagcode symbol value code symbol code symbol symbol {p₅-p₀} value valuevalue 001, 001 0 0 unspecified 100, 010 1 001, 010 1 0 0 000, 101 1 010,100 1 0 101, 000 1 010, 001 2 0 unspecified 100, 100 1 000, 011 3 0 000,110 1 011, 000 4 0 110, 000 1 001, 100 unused 010, 010 100, 001

The first position-coding pattern (“Yarrow”) uses the first eightregistration symbol values in Table 2 i.e. those registration symbolvalues mapping to a translation code symbol value of 0, 1 or 2. In otherwords, if the registration symbol value maps to a translation codesymbol value of 0, 1 or 2, then the position-coding pattern isidentified as the first position-coding pattern having 9 symbol groups303 contained in one tag 4A.

The additional translation code symbol values (i.e. 3 and 4) shown inTable 2 are reserved for the second position-coding pattern (“Saffron”)described in Section 4.6.1. Thus, if the registration symbol value mapsto a translation code symbol value of 3 or 4, then the position-codingpattern is identified as the second position-coding pattern having 4symbol groups 303 contained in one tag 4B. In this way, the registrationsymbol provides a means of distinguishing the first position-codingpattern from the second position-coding pattern. Subsequent decoding ofPPM data symbols proceeds in accordance with the position-coding patternidentified from decoding the registration symbol(s).

In the first position-coding pattern, each row of symbol groups and eachcolumn of symbol groups encodes a three-symbol 3-ary cyclic positioncode. (The Applicant's cyclic position codes are described in U.S. Pat.No. 7,082,562, the contents of which is herein incorporated byreference). The code consists of the codeword (0, 1, 2) and its cyclicshifts. The code has a minimum distance of 3, allowing a single symbolerror to be corrected. For each of the two orthogonal translations, thethree translation codes of an entire tag form a code with a minimumdistance of 9, allowing 4 symbol errors to be corrected. If additionalsymbols are visible within the field of view then they can be used foradditional redundancy.

The translation code symbol in the middle of the codeword (i.e. 1) ismapped to a set of 2-6PPM symbol values that are each other's reverse,while the two translation code symbols at the ends of the codeword (i.e.0 and 2) are each mapped to a set of 2-6PPM symbol values that are thereverses of the 2-6PPM symbol values in the other set. Thus a 0 readupside-down (i.e. rotated 180 degrees) becomes a 2, and vice versa,while a 1 read upside-down remains a 1. This allows translation to bedetermined independently of rotation.

Furthermore, in the first position-coding pattern, each 2-6PPM symbolvalue and its reverse map to opposite direction code symbol values(Table 2). The vertical registration symbols of an entire tag encode 9symbols of a vertical direction code. This has a minimum distance of 9,allowing 4 symbol errors to be corrected. The horizontal registrationsymbols of an entire tag encode 9 symbols of a horizontal directioncode. This has a minimum distance of 9, allowing 4 symbol errors to becorrected. If additional symbols are visible within the field of viewthen they can be used for additional redundancy. Any erasures detectedduring decoding of a translation code can also be used during decodingof a direction code, and vice versa. Together the orthogonal directioncodes allow the orientation of the tag to be determined.

The top left corner of an un-rotated tag is identified by a symbol groupwhose translation symbols are both zero and whose direction symbols areboth zero.

3.6.2 Active Area Flag Code

The flag symbol consists of one bit of data, and is encoded in some ofthe vertical and horizontal registration symbols, as shown in Table 2.

The flag symbol is unique to a tag and is therefore coded redundantly ineach quadrant of the tag. Since the flag symbol is encoded in eachregistration code symbol, it appears four times within each quadrant(assuming the central registration code symbols participate in eachquadrant, as usually supported by the minimum field of view). Foursymbols form a code with a minimum distance of 4, allowing 1 error to becorrected. If additional symbols are visible within the field of viewthen they can be used for additional redundancy. Any errors detectedduring decoding of translation and/or direction codes can also be usedto flag erasures during decoding of the flag code. Since the flag codeencodes the active area flag, it can meaningfully be interpreted as seteven if ambiguous.

3.6.3 Coordinate Data

The tag 4A contains an x-coordinate codeword and a y-coordinate codewordused to encode the x and y coordinates of the tag respectively. Thecodewords are of a shortened 2⁴-ary (11, 3) or (11, 5) Reed-Solomoncode. The tag therefore encodes either 12-bit or 20-bit coordinates. An(11, 5) code is used if the <region has long coordinates> flag in theregion flags is set (see Table 5). An (11, 3) code is used otherwise.

Each x coordinate codeword is replicated twice within the tag—in eachhorizontal half (“north” and “south”), and is constant within the columnof tags containing the tag. Likewise, each y coordinate codeword isreplicated twice within the tag—in each vertical half (“east” and“west”), and is constant within the row of tags containing the tag. Thisguarantees that an image of the tag pattern large enough to contain acomplete tag is guaranteed to contain a complete instance of eachcoordinate codeword, irrespective of the alignment of the image with thetag pattern. The instance of either coordinate codeword may consist offragments from different tags.

It should be noted that, in the first position-coding pattern, somecoordinate symbols are not replicated and are placed on the dividingline between the two halves of the tag. This arrangement saves tag spacesince there are not two complete replications of each x-coordinatecodeword and each y-coordinate codeword contained in a tag. Since thefield of view is at least three macrodot units larger than the tag (asdiscussed in Section 3.10), the coordinate symbols placed on thedividing line (having a width 2 macrodot units) are still captured whenthe surface is imaged. Hence, each interaction with the coded surfacestill provides the tag location.

The layout of the x-coordinate codeword is shown in FIG. 8. The layoutof the y-coordinate codeword is shown in FIG. 9. It can be seen thatx-coordinate symbols X4, X5, X6, X7, X8 and X9 are placed in a centralcolumn 310 of the tag 4A, which divides the eastern half of the tag fromthe western half. Likewise, the y-coordinate symbols Y4, Y5, Y6, Y7, Y8and Y9 are placed in a central row 312 of the tag 4A, which divides thenorthern half of the tag from the southern half.

The central column 310 and central row 312 each have a width q, whichcorresponds to a width of 2s, where s is the macrodot spacing.

3.6.4 Common Data

The tag 4A contains four codewords A, B, C and D which encodeinformation common to a set of contiguous tags in a surface region. TheA codeword is of a 2⁴-ary (15, 5) Reed-Solomon code. The B, C and Dcodewords are of a 2⁴-ary (15, 7) or (15, 9) Reed-Solomon code. The tagtherefore encodes either 112 or 136 bits of information common to a setof contiguous tags. A (15, 9) code is used for the B, C and D codewordsif the <region has a long region ID> flag in the region flags is set(see Table 6). A (15, 7) code is used otherwise.

The common codewords are replicated throughout a tagged region. Thisguarantees that an image of the tag pattern large enough to contain acomplete tag is guaranteed to contain a complete instance of each commoncodeword, irrespective of the alignment of the image with the tagpattern. The instance of each common codeword may consist of fragmentsfrom different tags.

The layout of the common codewords is shown in FIG. 10. The codewordshave the same layout, rotated 90 degree relative to each other.

3.6.5 Optional Data

The tag optionally contains a codeword E. This codeword may be used toencode a secret-key signature or a fragment of an embedded data object.These are discussed further in Sections 3.6.6 and Section 3.9.3respectively. The codeword is of a 2⁴-ary (15, 9) Reed-Solomon code.

The layout of the optional codeword is shown in FIG. 11.

3.6.6 Secret-Key Signature

The tag optionally contains an entire secret-key digital signaturecommon to a set of contiguous tags in a surface region. The signatureconsists of sixteen 2⁴-ary symbols (i.e. symbol E15 is also used). Thetag therefore optionally encodes up to 64 bits of secret-key signaturedata.

The signature is replicated throughout a tagged region. This guaranteesthat an image of the tag pattern large enough to contain a complete tagis guaranteed to contain a complete instance of the signature,irrespective of the alignment of the image with the tag pattern. Theinstance of the signature may consist of fragments from different tags.

The signature, if present, is encoded in the E codeword described inSection 3.6.5.

Digital signatures are discussed further in Section 3.9.4.

3.6.7 Complete Tag

FIG. 12 shows the layout of the data of a complete tag, with each symbolgroup comprising ten data symbols. The vertical and horizontalregistration symbols are not shown in FIG. 12.

3.7 Reed-Solomon Encoding 3.7.1 Reed-Solomon Codes

All data is encoded using a Reed-Solomon code defined over GF(2⁴). Thecode has a natural length n of 15. The dimension k of the code is chosento balance the error correcting capacity and data capacity of the code,which are (n−k)/2 and k symbols respectively.

The code may be punctured, by removing high-order redundancy symbols, toobtain a code with reduced length and reduced error correcting capacity.The code may also be shortened, by replacing high-order data symbolswith zeros, to obtain a code with reduced length and reduced datacapacity. Both puncturing and shortening can be used to obtain a codewith particular parameters. Shortening is preferred, where possible,since this avoids the need for erasure decoding.

The code has the following primitive polynominal:

p(x)=x ⁴ +x+1

The code has the following generator polynominal:

n−k

${g(x)} = {\prod\limits_{i = 1}\; \left( {x + \alpha^{i}} \right)}$

For a detailed description of Reed-Solomon codes, refer to Wicker, S. B.and V. K. Bhargava, eds., Reed-Solomon Codes and Their Applications,IEEE Press, 1994.

3.7.2 Codeword Organization

As shown in FIG. 13, redundancy coordinates r_(i) and data coordinatesd_(i) of the code are indexed from left to right according to the powerof their corresponding polynomial terms. The symbols X_(i) of a completecodeword are indexed from right to left to match the bit order of thedata. The bit order within each symbol is the same as the overall bitorder.

3.7.3 Code Instances

Table 3 defines the parameters of the different codes used in the tag.

TABLE 3 Codeword instances error- correcting data length dimensioncapacity capacity^(a) name description (n) (k) (symbols) (bits) X, Ycoordinate   11^(a) 3 4 12 codewords (see 5 3 20 Section 3.6.3) A firstcommon 15 5 5 20 codeword B, C, D other common 15 7 3 28 codewords 9 336 (see Section 3.6.4) E optional 15 9 3 36 codeword (see Section 3.6.5)^(a)shortened

3.7.4 Cyclic Redundancy Check

The region ID is protected by a 16-bit cyclic redundancy check (CRC).This provides an added layer of error detection after Reed-Solomon errorcorrection, in case a codeword containing a part of the region ID ismis-corrected.

The CRC has the following generator polynomial:

g(x)=x ¹⁶ +x ¹² +x ⁵+1

The CRC is initialised to 0xFFFF. The most significant bit of the regionID is treated as the most significant coefficient of the datapolynomial.

3.8 Tag Coordinate Space

The tag coordinate space has two orthogonal axes labelled x and yrespectively. When the positive x axis points to the right then thepositive y axis points down.

The surface coding does not specify the location of the tag coordinatespace origin on a particular tagged surface, nor the orientation of thetag coordinate space with respect to the surface. This information isapplication-specific. For example, if the tagged surface is a sheet ofpaper, then the application which prints the tags onto the paper mayrecord the actual offset and orientation, and these can be used tonormalise any digital ink subsequently captured in conjunction with thesurface.

The position encoded in a tag is defined in units of tags and is definedto be the centre of the top left target. The origin of a particular tagpattern is therefore the centre of the top left target of the tag thatencodes coordinate pair (0, 0).

The surface coding is optionally displaced from its nominal positionrelative to the surface by an amount derived from the region ID. Thisensures that the utilisation of a pagewidth digital printhead used toprint the surface coding is uniform. The displacement of the surfacecoding is negative, hence the displacement of the region described bythe surface coding is positive relative to the surface coding. Themagnitude of the displacement is the region ID modulo the width of thetag in 1600 dpi dots (i.e. 240). To accommodate non-1600 dpi printersthe actual magnitude of the displacement may vary from its nominal valueby up to half the dot pitch of the printer.

3.9 Tag Information Content 3.9.1 Field Definitions

Table 4 defines the information fields embedded in the firstposition-coding pattern.

TABLE 4 Field Definitions width field (bits) description unique to tagactive area flag 1 A flag indicating whether the area^(a) immediatelysurrounding a tag intersects an active area. x coordinate 12 or 20 Theunsigned x coordinate of the tag^(b). y coordinate 12 or 20 The unsignedy coordinate of the tag^(b). common to tagged region encoding 2 Theformat of the encoding. format 0: the present encoding. Other values arereserved region flags 10  Flags controlling the interpretation of region(see Table 5). coordinate 2 A value (p) indicating the precision of xand y precision coordinates according to the formula 8 + 4p. macrodot 4The ID of the macrodot size. size ID region ID 72 or 96 The ID of theregion containing the tags. CRC 16  A CRC of the region ID (see Section3.7.4) secret-key 64  An optional secret-key signature of the region.signature ^(a)the diameter of the area, centered on the tag, isnominally 2.5 times the diagonal size of the tag; this is to accommodatethe worst-case distance between the nib position and the imaged tag^(b)allows a coordinate value ranges of 14.8 m and 3.8 km for theminimum tag size of 3.6 mm (based on the minimum macrodot size of 120microns and 30 macrodots per tag)

An active area is an area within which any captured input should beimmediately forwarded to the corresponding Netpage server 10 forinterpretation. This also allows the Netpage server 10 to signal to theuser that the input has had an immediate effect. Since the server hasaccess to precise region definitions, any active area indication in thesurface coding can be imprecise so long as it is inclusive.

TABLE 5 Region flags bit meaning 0 Region is interactive, i.e. x andy-coordinates are present. 1 Region is active, i.e. the entire region isan active area. Otherwise active areas are identified by individualtags' active area flags. 2 Region ID is not serialized^(a). 3 Region hassecret-key signature (see Section 3.9.4) 4 Region has embedded data. 5Embedded data is a public-key signature (see Sections 3.9.3 and 3.9.4).6 Region has long coordinates^(b). 7 Region has a long region ID^(c). 8Region ID is an EPC. 9 Region is displaced according to region ID^(a)For an EPC this means that the serial number is replaced by a layoutnumber, to allow the package design associated with a product to varyover time (see US 2007/0108285, the contents of which is hereinincorporated by reference). ^(b)Hence the X and Y Reed-Solomon codewordshave less redundancy. ^(c)Hence, the B, C and D Reed-Solomon codewordshave less redundancy.

3.9.2 Mapping of Fields to Codewords

Table 6, Table 7 and Table 8 define how the information fields map tocodewords in the first position-coding pattern.

TABLE 6 Mapping of fields to coordinate codewords X and Y X and Ycodeword data codeword codeword field capacity field width field bitsbits X x coordinate 12 all all 20 Y y coordinate 12 all all 20

TABLE 7 Mapping of fields to common codewords A, B, C and D A, B, C andD codeword data codeword codeword field capacity field width field bitsbits A encoding any 2 all 1:0 format region 10 all 11:2  flags macrodot4 all 15:12 size ID region ID 28 4 71:68 19:16 36 95:92 B CRC any 16 all15:0  region ID 28 12 11:0  27:16 36 20 19:0  35:16 C region ID 28 39:12all 36 55:20 D region ID 28 67:40 all 36 91:56

TABLE 8 Mapping of fields to optional codeword E E codeword datacodeword codeword field capacity field width field bits bits E data 36  all all fragment secret-key 64^(a) all all digital signature ^(a)Entirecodeword (including 16^(th) symbol) is used for data i.e. there is noredundancy

As shown in Table 8, codeword E either contains a data fragment or asecret-key signature. These are described in Section 3.9.3 and Section3.9.4 respectively. The secret-key signature is present in a particulartag if the <region has secret-key signature> flag in the region flags isset, and the tag's active area flag is set. The data fragment is presentin a particular tag if the <region contains embedded data> flag in theregion flags is set and the tag does not already contain a secret-keysignature.

When the region flags indicate that a particular codeword is absent thenthe codeword is not coded in the tag pattern, i.e. there are nomacrodots representing the codeword. This applies to the X, Y and Ecodewords i.e. the X and Y codewords are present if the <region isinteractive> flag in the region flags is set. The E codeword is presentif a secret-key signature or data fragment is present.

3.9.3 Embedded Data Object

If the <region has embedded data> flag in the region flags is set thenthe surface coding contains embedded data. The embedded data is encodedin multiple contiguous tags' data fragments, and is replicated in thesurface coding as many times as it will fit.

The embedded data is encoded in such a way that a random and partialscan of the surface coding containing the embedded data can besufficient to retrieve the entire data. The scanning system reassemblesthe data from retrieved fragments, and reports to the user whensufficient fragments have been retrieved without error.

As shown in Table 9, each block has a data capacity of 176-bits. Theblock data is encoded in the data fragments of a contiguous group of sixtags arranged in a 3×2 rectangle.

The block parameters are as defined in Table 9. The E codeword of eachtag may encode a fragment of the embedded data.

TABLE 9 Block parameters parameter value description w 3 The width ofthe block, in tags h 2 The height of the block, in tags. b 176 The datacapacity of the block, in bits

If the E codeword of a particular tag does not contain a fragment of theembedded data, then the pen 400 can discover this implicitly by thefailure of the codeword to decode, or explicitly from the tag's activearea flag.

Data of arbitrary size may be encoded into a superblock consisting of acontiguous set of blocks, typically arranged in a rectangle. The size ofthe superblock may be encoded in each block.

The superblock is replicated in the surface coding as many times as itwill fit, including partially along the edges of the surface coding.

The data encoded in the superblock may include, for example, moreprecise type information, more precise size information, and moreextensive error detection and/or correction data.

3.9.4 Digital Signatures

As described in Section 3.6.6, a region may contain a digital signature.

If the <region has a secret-key signature> flag in the region flags isset, then the region has a secret-key digital signature. In an onlineenvironment the secret-key signature can be verified, in conjunctionwith the region ID, by querying a server with knowledge of thesecret-key signature or the corresponding secret key.

If the region contains embedded data and the <embedded data is apublic-key signature> flag in the region flag is set, then the surfacecoding contains an embedded public-key digital signature of the regionID.

In an online environment any number of signature fragments can be used,in conjunction with the region ID and optionally the secret-keysignature, to validate the public-key signature by querying a serverwith knowledge of the full public-key signature or the correspondingprivate key.

In an offline (or online) environment the entire public-key signaturecan be recovered by reading multiple tags, and can then be verifiedusing the corresponding public signature key. The actual length and typeof the signature are determined from the region ID during signaturevalidation i.e. typically from a previously-retrieved digital signatureassociated with a sequence of region IDs.

Digital signature verification is discussed in the Applicant's USPublication No. 2007/0108285, the contents of which are hereinincorporated by reference.

3.10 Tag Imaging

The minimum imaging field of view required to guarantee acquisition ofdata from an entire tag 4A has a diameter of 46.7s (i.e. ((3×10)+3)√2s),allowing for arbitrary rotation and translation of the surface coding inthe field of view. Notably, the imaging field of view does not have tobe large enough to guarantee capture of an entire tag—the arrangement ofthe data symbols within each tag ensures that a any square portion oflength (l+3s) captures the requisite information in full, irrespectiveof whether a whole tag is actually visible in the field-of-view. As usedherein, l is defined as the length of a tag.

In terms of imaging the coding pattern, the imaging field-of-view istypically a circle. Accordingly, the imaging field-of-view shouldpreferably have diameter of at least (l+3s)√2 and less than two tagdiameters. Importantly, the field-of-view is not required to be at leasttwo tag diameters, in contrast with prior art tag designs, because it isnot essential to capture an entire tag 4A in the field of view.

The extra three macrodot units ensure that pulse-position modulatedvalues can be decoded from spatially coherent samples. Furthermore, theextra three macrodot units ensure that all requisite data symbols 304Acan be read with each interaction. These include the coordinate symbolsfrom a central column or row of a tag (see Section 3.6.3) having a widthof 2s.

In the present context, a “tag diameter” is given to mean the length ofa tag diagonal.

Given a maximum macrodot spacing of 127 microns, this gives a requiredfield of view of 5.93 mm.

4 Second Position-Coding Pattern (“Saffron”) 4.1 Background

As will be appreciated from the following description, the secondposition-coding pattern bears many similarities with the first positioncoding pattern. The most notable difference is that each tag comprises 4rather than 9 symbol groups 303. Furthermore, the registration symbolsin the second position-coding pattern map to translation code symbolvalues (3, 4) specifically identifying the second position-codingpattern, as described earlier in Section 3.6.1. The complete secondposition-coding pattern will now be described in detail below.

4.2 General Tag Structure

In common with the first position-coding pattern, each tag 4B of thesecond position-coding pattern is represented by two kinds of elements.Referring to FIGS. 14 and 15, the first kind of element is a targetelement. Target elements in the form of target dots 301 allow the tag 4Bto be located in an image of a coded surface, and allow the perspectivedistortion of the tag to be inferred. The second kind of element is adata element in the form of a macrodot 302 (see FIG. 6). The macrodots302 encode data values.

FIG. 14 shows the structure of a complete tag 4B from the secondposition-coding pattern, with target elements 301 shown. The tag 4B issquare and contains nine target elements. Those target elements 301located at the edges and corners of the tag (eight in total) are sharedby adjacent tags and define the perimeter of the tag 4B.

The tag 4B consists of a square array of four symbol groups 303. Symbolgroups 303 are demarcated by the target elements 301 so that each symbolgroup is contained within a square defined by four target elements.Adjacent symbol groups 303 are contiguous and share targets.

Since the target elements 301 are all identical, they do not demarcateone tag from its adjacent tags. Viewed purely at the level of targetelements, only symbol groups 303, which define cells of a target grid,can be distinguished—the tags 4B themselves are indistinguishable byviewing only the target elements. Hence, tags 4B must be aligned withthe target grid as part of tag decoding.

The tag 4B is designed to allow all tag data to be recovered from animaging field of view substantially the size of the tag.

4.3 Symbol Groups

As shown in FIG. 15, each of the nine symbol groups 303 comprises eightdata symbols 304B, each data symbol being part of a codeword. Inaddition, each symbol group 303 comprises a pair of registrationsymbols—a vertical registration symbol (‘VRS’) and a horizontalregistration symbol (‘HRS’). These allow the orientation and/ortranslation of the tag 4B in the field of view to be determined.

Each data symbol 304B is a multi-pulse position modulated (PPM) datasymbol. Typically, each PPM data symbol 304B encodes 5-bits using 2-9PPMencoding. i.e. 2 macrodots in any of 9 positions {p₀, p₁, p₂, p₃, p₄,p₅, p₆, p₇, p₈}.

FIG. 16 shows the layout for a 2-9PPM data symbol 304B.

Each symbol group also contains a 2-6PPM vertical registration symbol(VRS) and a 2-6PPM horizontal registration symbol (HRS), as described inSections 3.3 and 3.6.1 above.

Table 10 defines the mapping from 2-6PPM symbol values to data symbolvalues. Unused symbol values can be treated as erasures.

TABLE 10 2-9PPM symbol to data symbol value mapping 2-9PPM symbol valuedata symbol value (p₈-p₀) (base 16) 000, 000, 011  0 000, 000, 101  1000, 000, 110  2 000, 001, 001  3 000, 001, 010  4 000, 001, 100  5 000,010, 001  6 000, 010, 010  7 000, 010, 100  8 000, 011, 000  9 000, 100,001 a 000, 100, 010 b 000, 100, 100 c 000, 101, 000 d 000, 110, 000 e001, 000, 001 f 001, 000, 010 10 001, 000, 100 11 001, 001, 000 12 001,010, 000 13 001, 100, 000 14 010, 000, 001 15 010, 000, 010 16 010, 000,100 17 010, 001, 000 18 010, 010, 000 19 010, 100, 000 1a 011, 000, 0001b 100, 000, 001 1c 100, 000, 010 1d 100, 000, 100 1e 100, 001, 000 1f100, 010, 000 unused 100, 100, 000 unused 101, 000, 000 unused 110, 000,000 unused

4.4 Targets and Macrodots

The spacing of macrodots 302 in both dimensions, as shown in FIG. 6, isspecified by the parameter s. In the second position-coding pattern, ithas a nominal value of 159 μm, based on 10 dots printed at a pitch of1600 dots per inch.

A macrodot 302 is nominally square with a nominal size of (5/10)s.However, it is allowed to vary in size by ±10% according to thecapabilities of the device used to produce the pattern.

A target 301 is nominally circular with a nominal diameter of (15/10)s.However, it is allowed to vary in size by ±10% according to thecapabilities of the device used to produce the pattern.

Each symbol group 303 has a width of 10s. Therefore, each tag 4B has awidth of 20s and a length of 20s. However, it should be noted from FIG.15 that the tag 4B is configured so that some data symbols 304 extendbeyond the perimeter edge of the tag 4B by one macrodot unit (1s), andinterlock with complementary data symbols from adjacent tags. Thisarrangement provides a tessellated pattern of data symbols 304A withinthe target grid. From a data acquisition standpoint, tessellation ofdata symbols in this way increases the effective length of each tag 4Bby one macrodot unit.

The macrodot spacing, and therefore the overall scale of the tagpattern, is allowed to vary between 152 μm and 169 μm according to thecapabilities of the device used to produce the pattern. Any deviationfrom the nominal scale is recorded in each tag (via a macrodot size IDfield) to allow accurate generation of position samples.

These tolerances are independent of one another. They may be refinedwith reference to particular printer characteristics.

4.5 Field of View

As mentioned above, the tag 4B is designed to allow all tag data to berecovered from an imaging field of view roughly the size of the tag.

Although data which is common to a set of tags, in one or both spatialdimensions, may be decoded from fragments from adjacent tags,pulse-position modulated values are best decoded from spatially-coherentsamples (i.e. from a whole symbol as opposed to partial symbols atopposite sides of the field of view), since this allows raw samplevalues to be compared without first being normalised. This implies thatthe field of view must be large enough to contain two complete copies ofeach such pulse-position modulated value. The tag is designed so thatthe maximum extent of a pulse-position modulated value is fourmacrodots. Making the field of view at least as large as the tag plusfour macrodot units guarantees that pulse-position modulated values canbe coherently sampled.

4.6 Encoded Codes and Codewords

In this section (Section 4.6), each symbol in FIGS. 17 to 20 is shownwith a unique label. The label consists of an alphabetic prefix whichidentifies which codeword the symbol is part of, and a numeric suffixwhich indicates the index of the symbol within the codeword. Forsimplicity only data symbols 304B are shown in FIGS. 17 to 20, notregistration symbols.

Although some symbol labels are shown rotated to indicate the symmetryof the layout of certain codewords, the layout of each symbol isdetermined by its position within a symbol group and not by the rotationof the symbol label (as described in, for example, the Applicant's USPublication No. 2006/146069).

4.6.1 Registration Symbols

Each registration symbol of the second position-coding pattern isencoded using 2-6PPM, as described above in Section 3.6.1 and FIG. 7.Furthermore, each registration symbol of the second position-codingpattern is positioned and configured in the same way as eachregistration symbol of the first position-coding pattern. However, thesecond position-coding pattern utilizes only those registration symbolvalues mapping to the translation code symbol values (3, 4). Thisenables the registration symbol to identify the second position-codingpattern, and distinguish it from the first position-coding pattern.

In other words, if the registration symbol value maps to a translationcode symbol value of 3 or 4, then the position-coding pattern isidentified as the second position-coding pattern having 4 symbol groups304B contained in one tag 4B.

In the first position-coding pattern, each row of symbol groups and eachcolumn of symbol groups encodes a two-symbol 2-ary cyclic position code.(The Applicant's cyclic position codes are described in U.S. Pat. No.7,082,562, the contents of which is herein incorporated by reference).The code consists of the codeword (3, 4) and its cyclic shifts. For eachof the two orthogonal translations, the two translation codes of anentire tag form a code with a minimum distance of 4, allowing 1 symbolerror to be corrected. If additional symbols are visible within thefield of view then they can be used for additional redundancy.

The two translation code symbols (3 and 4) are each mapped to a set of2-6PPM symbol values that are the reverses of the 2-6PPM symbol valuesin the other set. Thus a 3 read upside-down (i.e. rotated 180 degrees)becomes a 4, and vice versa. This allows translation to be determinedindependently of rotation.

Furthermore, in the first position-coding pattern, each 2-6PPM symbolvalue and its reverse map to opposite direction code symbol values(Table 2). The vertical registration symbols of an entire tag encode 4symbols of a vertical direction code. This has a minimum distance of 4,allowing 1 symbol error to be corrected. The horizontal registrationsymbols of an entire tag encode 4 symbols of a horizontal directioncode. This has a minimum distance of 4, allowing 1 symbol error to becorrected. If additional symbols are visible within the field of viewthen they can be used for additional redundancy. Any erasures detectedduring decoding of a translation code can also be used during decodingof a direction code, and vice versa. Together the orthogonal directioncodes allow the orientation of the tag to be determined.

The top left corner of an un-rotated tag is identified by a symbol groupwhose translation symbols are both zero and whose direction symbols areboth zero.

Although as shown in Table 2, the 2-6PPM registration symbol does notallow flag codes for the second position-coding pattern, it will beappreciated that a 3-6PPM registration symbol mapping to 20 availablesymbol values would allow the second position-coding pattern to containflag codes, if desired. In this case, 12 registration symbol values(3×2×2) would be used for the first position-coding pattern and 8registration symbols value (2×2×2) would be used for the secondposition-coding pattern.

4.6.2 Coordinate Data

The tag 4B contains an x-coordinate codeword and a y-coordinate codewordused to encode the x and y coordinates of the tag respectively. Thecodewords are of a shortened 2⁵-ary (4, 2) Reed-Solomon code. The tagtherefore encodes 10-bit coordinates.

Each x coordinate codeword is constant within the column of tagscontaining the tag. Likewise, each y coordinate codeword is constantwithin the row of tags containing the tag.

It should be noted that, in the second position-coding pattern, none ofthe coordinate symbols are replicated. Instead, all coordinate symbolsare placed in either one column or one row of the tag. This arrangementsaves tag space since it obviates the requirement for each tag tocontain two complete replications of each x-coordinate codeword and eachy-coordinate codeword. Since the field of view is at least four macrodotunits larger than the length of the tag, the coordinate symbols placedin a column or row line having a width of three macrodot units are stillcaptured when the surface is imaged. Hence, each interaction with thecoded surface still provides the tag location. The instance of eithercoordinate codeword may consist of fragments from different tags.

FIG. 17 shows the layout of an x-coordinate codeword X. The outline ofthe codeword X is shown in bold. It should be noted that the entirex-coordinate codeword is encoded by data symbols X0, X1, X2 and X3contained in a single column 313 of width v. Likewise, as shown in FIG.18, the entire y-coordinate codeword is encoded by data symbols Y0, Y1,Y2 and Y3 contained in a single row 315 of width v.

The column of x-coordinate symbols and the row of y-coordinate symbolseach have a width v, which corresponds to a width of 3s, where s is themacrodot spacing. Provided that an imaged portion of the secondposition-coding pattern contains a square of length (l+v), where l isthe length of the tag, then the imaged portion is guaranteed to containthe x-coordinate codeword and the y-coordinate codeword.

4.6.3 Common Data

The tag 4B contains one codeword A which encodes information common to aset of contiguous tags in a surface region. The A codeword is of ashortened 2⁵-ary (24, 16) Reed-Solomon code. The tag 4B thereforeencodes 80 bits of information common to a set of contiguous tags.

The common codeword is replicated throughout a tagged region. Thisguarantees that an image of the tag pattern large enough to contain acomplete tag is guaranteed to contain a complete instance of the commoncodeword, irrespective of the alignment of the image with the tagpattern. The instance of the common codeword may consist of fragmentsfrom different tags.

The layout of the common codeword is shown in FIG. 19.

4.6.3 Complete Tag

FIG. 20 shows the layout of the data of a complete tag 4B, with eachsymbol group comprising eight data symbols. The vertical and horizontalregistration symbols are not shown in FIG. 20.

4.7 Reed-Solomon Encoding 4.7.1 Reed-Solomon Codes

All data of the second position-coding pattern is encoded using aReed-Solomon code defined over GF(2⁵). The code has a natural length nof 31. The dimension k of the code is chosen to balance the errorcorrecting capacity and data capacity of the code, which are (n−k)/2 andk symbols respectively.

The code may be punctured, by removing high-order redundancy symbols, toobtain a code with reduced length and reduced error correcting capacity.The code may also be shortened, by replacing high-order data symbolswith zeros, to obtain a code with reduced length and reduced datacapacity. Both puncturing and shortening can be used to obtain a codewith particular parameters. Shortening is preferred, where possible,since this avoids the need for erasure decoding.

The code has the following primitive polynominal:

p(x)=x ⁵ +x ²+1

The code has the following generator polynominal:

${g(x)} = {\prod\limits_{i = 1}^{n - k}\; \left( {x + \alpha^{i}} \right)}$

For a detailed description of Reed-Solomon codes, refer to Wicker, S. B.and V. K. Bhargava, eds., Reed-Solomon Codes and Their Applications,IEEE Press, 1994.

4.7.2 Codeword Organization

As shown in FIG. 21, redundancy coordinates r_(i) and data coordinatesd_(i) of the code are indexed from left to right according to the powerof their corresponding polynomial terms. The symbols X_(i) of a completecodeword are indexed from right to left to match the bit order of thedata. The bit order within each symbol is the same as the overall bitorder.

4.7.3 Code Instances

Table 11 defines the parameters of the different codes used in the tag4B.

TABLE 11 Codeword instances error- correcting data length dimensioncapacity capacity^(a) name description (n) (k) (symbols) (bits) X, Ycoordinate  4^(a) 2 1 10 codewords (see Section 4.6.3) A first common24^(a) 16 8 80 codeword ^(a)shortened

4.7.4 Cyclic Redundancy Check

The region ID is protected by a 16-bit cyclic redundancy check (CRC).This provides an added layer of error detection after Reed-Solomon errorcorrection, in case a codeword containing a part of the region ID ismis-corrected.

The CRC has the following generator polynomial:

g(x)=x ¹⁶ +x ¹² +x ⁵+1

The CRC is initialised to 0xFFFF. The most significant bit of the regionID is treated as the most significant coefficient of the datapolynomial.

4.8 Tag Coordinate Space

The tags 4B of the second position-coding pattern use a coordinate spacecorresponding to the first position-coding pattern having two orthogonalaxes labelled x and y respectively. For a further discussion, seeSection 3.8 above.

4.9 Tag Information Content 4.9.1 Field Definitions

Table 12 defines the information fields embedded in the secondposition-coding pattern.

TABLE 12 Field Definitions width field (bits) description unique to tagx coordinate 10 The unsigned x coordinate of the tag^(a). y coordinate10 The unsigned y coordinate of the tag^(a). common to tagged regionencoding 2 The format of the encoding. format 0: the present encoding.Other values are reserved region flags 6 Flags controlling theinterpretation of region data (see Table 13). macrodot 4 The ID of themacrodot size. size ID region ID 52 The ID of the region containing thetags. CRC 16 A CRC of the region ID (see Section 4.7.4) ^(a)allows acoordinate value ranges of 3.1 m for the minimum tag size of 3.04 mm(based on the minimum macrodot size of 152 microns and 20 macrodots pertag)

TABLE 13 Region flags bit meaning 0 Region is interactive, i.e. x andy-coordinates are present. 1 Region is active, i.e. the entire region isan active area. 2 Region ID is serialized 3 Region is displacedaccording to region ID other Reserved for future use.

4.9.2 Mapping of Fields to Codewords

Tables 14 and 15 define how the information fields map to codewords inthe second position-coding pattern.

TABLE 14 Mapping of fields to coordinate codewords X and Y X and Ycodeword data codeword codeword field capacity field width field bitsbits X x coordinate 10 all all Y y coordinate 10 all all

TABLE 15 Mapping of fields to common codewords A, B, C and D codewordcodeword field field width field bits bits A CRC 16 all 15:0  region ID52 all 67:16 encoding format 2 all 69:68 region flags 6 all 75:70macrodot size ID 4 all 79:76

When the region flags indicate that a particular codeword is absent thenthe codeword is not coded in the tag pattern, i.e. there are nomacrodots representing the codeword. This applies to the X and Y i.e.the X and Y codewords are present if the <region is interactive> flag inthe region flags is set.

4.10 Tag Imaging

The minimum imaging field of view required to guarantee acquisition ofdata from an entire tag 4B has a diameter of 33.9s (i.e. ((2×10)+4)√2s),allowing for arbitrary rotation and translation of the surface coding inthe field of view. Notably, the imaging field of view does not have tobe large enough to guarantee capture of an entire tag—the arrangement ofthe data symbols within each tag ensures that a any square portion oflength (l+4s) captures the requisite information in full, irrespectiveof whether a whole tag is actually visible in the field-of-view. As usedherein, l is defined as the length of a tag.

In terms of imaging the coding pattern, the imaging field-of-view istypically a circle. Accordingly, the imaging field-of-view shouldpreferably have diameter of at least (l+4s)√2 and less than two tagdiameters. Importantly, the field-of-view is not required to be at leasttwo tag diameters, in contrast with prior art tag designs, because it isnot essential to capture an entire tag 4B in the field of view.

The extra four macrodot units ensure that pulse-position modulatedvalues can be decoded from spatially coherent samples. Furthermore, theextra four macrodot units ensure that all requisite data symbols 304Bcan be read with each interaction. These include the coordinate symbolsfrom a column or row of a tag (see Section 4.6.2) having a width of 3s.

In the present context, a “tag diameter” is given to mean the length ofa tag diagonal.

Given a maximum macrodot spacing of 169 microns, this gives a requiredfield of view of 5.74 mm.

Thus, a field of view of at least 5.93 mm (see Section 3.10) issufficient to capture data from an entire tag 4A from the firstposition-coding pattern or an entire tag 4B from the secondposition-coding pattern. Self-evidently, the requisite field of view forcapturing either tag 4A or tag 4B will vary depending on the macrodotspacing in either the first or second position coding patterns. This, inturn, depends on the print resolution of a printer used to print therespective position-coding pattern.

4.11 Tag Decoding

FIG. 22 shows a tag image processing and decoding process flow up to thestage of sampling registration symbols and decoding the translationcodewords. Firstly, a raw image 802 of the tag pattern is acquired (at800), for example via an image sensor such as a CCD image sensor, CMOSimage sensor, or a scanning laser and photodiode image sensor. The rawimage 802 is then typically enhanced (at 804) to produce an enhancedimage 806 with improved contrast and more uniform pixel intensities.Image enhancement may include global or local range expansion,equalization, and the like. The enhanced image 806 is then typicallyfiltered (at 808) to produce a filtered image 810. Image filtering mayconsist of low-pass filtering, with the low-pass filter kernel sizetuned to obscure macrodots 302 but to preserve targets 301. Thefiltering step 808 may include additional filtering (such as edgedetection) to enhance target features 301. Encoding of data symbols 304using pulse position modulation (PPM) provides a more uniform codingpattern 3 than simple binary dot encoding (as described in, for example,U.S. Pat. No. 6,832,717). Advantageously, this helps separate targets301 from data areas, thereby allowing more effective low-pass filteringof the PPM-encoded data compared to binary-coded data.

Following low-pass filtering, the filtered image 810 is then processed(at 812) to locate the targets 301. This may consist of a search fortarget features whose spatial inter-relationship is consistent with theknown geometry of the tag pattern (i.e. targets positioned at thecorners of square cells). Candidate targets may be identified directlyfrom maxima in the filtered image 810, or may be the subject of furthercharacterization and matching, such as via their (binary or grayscale)shape moments (typically computed from pixels in the enhanced image 806based on local maxima in the filtered image 810), as described in U.S.Pat. No. 7,055,739, the contents of which is herein incorporated byreference.

The identified targets 301 are then assigned (at 816) to a target grid818. Each cell of the grid 818 contains a symbol group 303, and severalsymbol groups will of course be visible in the image. At this stage,individual tags 4 will not be identifiable in the target grid 818, sincethe targets 301 do not themselves demarcate one tag from another.

To allow macrodot values to be sampled accurately, the perspectivetransform of the captured image must be inferred. Four of the targets301 are taken to be the perspective-distorted corners of a square ofknown size in tag space, and the eight-degree-of-freedom perspectivetransform 822 is inferred (at 820), based on solving the well-understoodequations relating the four tag-space and image-space point pairs.Calculation of the 2D perspective transform is described in detail in,for example, Applicant's U.S. Pat. No. 6,832,717, the contents of whichis herein incorporated by reference.

Since each image of either the first or second position-coding patternwill typically contain at least 9 targets arranged in a square grid, theaccuracy of calculating the 2D perspective transform is improvedcompared to the Applicant's previous tag designs described in, forexample, U.S. Pat. No. 6,832,717. Hence, more accurate positioncalculation can be achieved with the tag design of the presentinvention.

The inferred tag-space to image-space perspective transform 822 is usedto project each known macrodot position in tag space into image space.Since all bits in the tags are represented by PPM-encoding, the presenceor absence of each macrodot 302 can be determined using a localintensity reference, rather than a separate intensity reference. Thus,PPM-encoding provides improved data sampling compared with pure binaryencoding.

The next stage determines a type of position-coding pattern being imagedby the pen 400 from a translation codeword. In other words, this stagedistinguishes the first position-coding pattern from the secondposition-coding pattern for subsequent sampling and decoding.

Two or more orthogonal registration symbols (‘VRS’ and ‘HRS’) aresampled (at 824), to allow decoding of the orthogonal translationcodewords and the orthogonal direction codewords. A flag symbol valuemay also be decoded subsequently from the decoded registration symbols.

Decoding of the orthogonal translation codewords (at 828) yields eithera (0, 1, 2) translation codeword or a (3, 4) translation codeword (at830).

Referring now to FIG. 23, the (0, 1, 2) translation codeword indicatesnine symbol groups per tag, thereby identifying (at 832A) the imagedposition-coding pattern as being the first position-coding pattern(“Yarrow”) containing tags 4A. Alternatively, the (3, 4) translationcodeword indicates four symbol groups per tag, thereby identifying (at832B) the imaged position-coding pattern as being the secondposition-coding pattern (“Saffron”) containing tags 4B.

Once the position-coding pattern has been identified at 832A or 832B,subsequent sampling and decoding proceeds in accordance with theposition-coding pattern thus identified. Accordingly, the decodedorthogonal translation codewords are used to determine the translationof tags(s) in the field of view relative to the target grid 818. Thisenables alignment of the tags 4A or 4B with the target grid 818, therebyallowing individual tag(s), or portions thereof, to be distinguished inthe coding pattern 3 in the field of view. In the case of the firstposition-coding pattern, the tags 4A (each containing nine symbolgroups) are aligned (at 834A) with the target grid 818. In the case ofthe second position-coding pattern, the tags 4B (each containing foursymbol groups) are aligned (at 834B) with the target grid 818.

Since each symbol group 303 contains orthogonal registration symbols,multiple translation codes can be decoded to provide robust translationdetermination. As described in Sections 3.6.1 and 4.6.1, the translationcode is a cyclic position code. Since each row and each column of a tagcontains M symbol groups, the code has minimum distance M×M. This allowsrobust determination of the alignment of tags 4A or 4B with the targetgrid 818. The alignment needs to be both robust and accurate since thereare many possible alignments when each tag contains multiple symbolgroups 303.

After the translation of symbol groups 303 relative to tags 4A or 4B hasbeen determined, then at least two orthogonal direction codes aredecoded (at 836A or 836B) to provide the orientation 838A or 838B. Asdescribed in Sections 3.6.1 and 4.6.1, since N vertical registrationsymbols in a tag form a vertical direction code with minimum distance N,the vertical direction code is capable of correcting (N−1)/2 errors. Thehorizontal direction code is similarly capable of correcting (N−1)/2errors using N horizontal registration symbols. Hence, orientationdetermination is very robust and capable of correcting errors, dependingon the number of registration symbols sampled.

Once initial imaging and decoding has yielded the 2D perspectivetransform, the orientation, and the translation of tag(s) relative tothe target grid, the data codewords can then be sampled and decoded (at840A or 840B) to yield the requisite decoded codewords 842A or 842B.

Decoding of data codewords in the first position-coding pattern(“Yarrow”) typically proceeds as follows:

-   -   sample and decode Reed-Solomon codeword containing encoding        format etc. (A)    -   determine encoding format, and reject unknown encoding    -   on decode error flag bad region ID sample    -   determine region ID Reed-Solomon codeword format from region        flags    -   sample and decode Reed-Solomon codeword containing region ID (B,        C and D)    -   verify CRC of region ID    -   on decode error flag bad region ID sample    -   determine region ID    -   determine x and y coordinate Reed-Solomon codeword format from        region flags    -   sample and decode x and y coordinate Reed-Solomon codewords (X        and Y)    -   determine tag x-y location from codewords    -   determine nib x-y location from tag x-y location and perspective        transform taking into account macrodot size (from macrodot size        ID)    -   decode four or more flag symbols to determine active area flag    -   determine active area status of nib location with reference to        active area flag    -   encode region ID, nib x-y location, and nib active area status        in digital ink (“interaction data”)

Decoding of data codewords in the second position-coding pattern(“Saffron”) typically proceeds as follows:

-   -   sample and decode common Reed-Solomon codeword (A)    -   determine encoding format, and reject unknown encoding    -   on decode error flag bad format sample    -   determine region ID Reed-Solomon codeword format from region        flags    -   verify CRC of region ID    -   on decode error flag bad region ID sample    -   determine region ID    -   sample and decode x and y coordinate Reed-Solomon codewords (X        and Y)    -   determine tag x-y location from codewords    -   determine nib x-y location from tag x-y location and perspective        transform taking into account macrodot size (from macrodot size        ID)    -   encode region ID and nib x-y location in digital ink        (“interaction data”)

In practice, when decoding a sequence of images of a tag pattern, it isuseful to exploit inter-frame coherence to obtain greater effectiveredundancy.

Region ID decoding need not occur at the same rate as position decoding.

The skilled person will appreciate that the decoding sequence describedabove represents one embodiment of the present invention. It will, ofcourse, be appreciated that the interaction data sent from the pen 400to the netpage system may include other data e.g. digital signature (seeSection 3.9.4), pen mode (see US 2007/125860 incorporated herein byreference), orientation data, force data, pen ID, nib ID etc.

An example of interpreting interaction data, received by the netpagesystem from the netpage pen 400, is discussed briefly above inSection 1. A more detailed discussion of how the netpage system mayinterpret interaction data can be found in the Applicant'spreviously-filed applications (see, for example, US 2007/130117 and US2007/108285, the contents of which are herein incorporated byreference).

5. Netpage Pen 5.1 Functional Overview

The active sensing device (or “reader”) of the netpage system may takethe form of a clicker (for clicking on a specific position on asurface), a pointer having a stylus (for pointing or gesturing on asurface using pointer strokes), or a pen having a marking nib (formarking a surface with ink when pointing, gesturing or writing on thesurface). For a description of various netpage readers, reference ismade to U.S. Pat. No. 7,105,753; U.S. Pat. No. 7,015,901; U.S. Pat. No.7,091,960; and US Publication No. 2006/0028459, the contents of each ofwhich are herein incorporated by reference.

It will be appreciated that the present invention may utilize anysuitable optical reader. However, the Netpage pen 400 will be describedherein as one such example.

In accordance with the present invention, either the firstposition-coding pattern (as described in Section 3) or the secondposition-coding pattern (as described in Section 4) may be read usingthe same Netpage pen 400 using the image processing and decoding stepsdescribed in Section 4.11.

The Netpage pen 400 is a motion-sensing writing instrument which worksin conjunction with a tagged Netpage surface containing either the firstor second position-coding patterns. The pen 400 incorporates aconventional ballpoint pen cartridge for marking the surface, an imagesensor and processor for simultaneously capturing the absolute path ofthe pen on the surface and identifying the surface, a force sensor forsimultaneously measuring the force exerted on the nib, and a real-timeclock for simultaneously measuring the passage of time.

While in contact with a tagged surface, as indicated by the forcesensor, the pen continuously images the surface region adjacent to thenib, and decodes the nearest tag in its field of view to determine boththe identity of the surface, its own instantaneous position on thesurface and the pose of the pen. The pen thus generates a stream oftimestamped position samples relative to a particular surface, andtransmits this stream to the Netpage server 10. The sample streamdescribes a series of strokes, and is conventionally referred to asdigital ink (DInk). Each stroke is delimited by a pen down and a pen upevent, as detected by the force sensor. More generally, any dataresulting from an interaction with a Netpage, and transmitted to theNetpage server 10, is referred to herein as “interaction data”.

The pen samples its position at a sufficiently high rate (nominally 100Hz) to allow a Netpage server to accurately reproduce hand-drawnstrokes, recognise handwritten text, and verify hand-written signatures.

The Netpage pen also supports hover mode in interactive applications. Inhover mode the pen is not in contact with the paper and may be somesmall distance above the surface of the paper (or other substrate). Thisallows the position of the pen, including its height and pose to bereported. In the case of an interactive application the hover modebehaviour can be used to move a cursor without marking the paper, or thedistance of the nib from the coded surface could be used for toolbehaviour control, for example an air brush function.

The pen includes a Bluetooth radio transceiver for transmitting digitalink via a relay device to a Netpage server. When operating offline froma Netpage server the pen buffers captured digital ink in non-volatilememory. When operating online to a Netpage server the pen transmitsdigital ink in real time.

The pen is supplied with a docking cradle or “pod”. The pod contains aBluetooth to USB relay. The pod is connected via a USB cable to acomputer which provides communications support for local applicationsand access to Netpage services.

The pen is powered by a rechargeable battery. The battery is notaccessible to or replaceable by the user. Power to charge the pen can betaken from the USB connection or from an external power adapter throughthe pod. The pen also has a power and USB-compatible data socket toallow it to be externally connected and powered while in use.

The pen cap serves the dual purpose of protecting the nib and theimaging optics when the cap is fitted and signaling the pen to leave apower-preserving state when uncapped.

5.2 Ergonomics and Layout

FIG. 24 shows a rounded triangular profile giving the pen 400 anergonomically comfortable shape to grip and use the pen in the correctfunctional orientation. It is also a practical shape for accommodatingthe internal components. A normal pen-like grip naturally conforms to atriangular shape between thumb 402, index finger 404 and middle finger406.

As shown in FIG. 25, a typical user writes with the pen 400 at a nominalpitch of about 30 degrees from the normal toward the hand 408 when held(positive angle) but seldom operates a pen at more than about 10 degreesof negative pitch (away from the hand). The range of pitch angles overwhich the pen 400 is able to image the pattern on the paper has beenoptimised for this asymmetric usage. The shape of the pen 400 helps toorient the pen correctly in the user's hand 408 and to discourage theuser from using the pen “upside-down”. The pen functions “upside-down”but the allowable tilt angle range is reduced.

The cap 410 is designed to fit over the top end of the pen 400, allowingit to be securely stowed while the pen is in use. Multi colour LEDsilluminate a status window 412 in the top edge (as in the apex of therounded triangular cross section) of the pen 400 near its top end. Thestatus window 412 remains un-obscured when the cap is stowed. Avibration motor is also included in the pen as a haptic feedback system(described in detail below).

As shown in FIG. 26, the grip portion of the pen has a hollow chassismolding 416 enclosed by a base molding 528 to house the othercomponents. The ink cartridge 414 for the ball point nib (not shown)fits naturally into the apex 420 of the triangular cross section,placing it consistently with the user's grip. This in turn providesspace for the main PCB 422 in the centre of the pen and for the battery424 in the base of the pen. By referring to FIG. 27A, it can be seenthat this also naturally places the tag-sensing optics 426 unobtrusivelybelow the nib 418 (with respect to nominal pitch). The nib molding 428of the pen 400 is swept back below the ink cartridge 414 to preventcontact between the nib molding 428 and the paper surface when the penis operated at maximum pitch.

As best shown in FIG. 27B, the imaging field of view 430 emerges througha centrally positioned IR filter/window 432 below the nib 418, and twonear-infrared illumination LEDs 434, 436 emerge from the two bottomcorners of the nib molding 428. Each LED 434, 436 has a correspondingillumination field 438, 440.

As the pen is hand-held, it may be held at an angle that causesreflections from one of the LED's that are detrimental to the imagesensor. By providing more than one LED, the LED causing the offendingreflections can be extinguished.

Specific details of the pen mechanical design can be found in USPublication No. 2006/0028459, the contents of which are hereinincorporated by reference.

5.3 Pen Feedback Indications

FIG. 28 is a longitudinal cross section through the centre-line if thepen 400 (with the cap 410 stowed on the end of the pen). The penincorporates red and green LEDs 444 to indicate several states, usingcolours and intensity modulation. A light pipe 448 on the LEDs 444transmit the signal to the status indicator window 412 in the tubemolding 416. These signal status information to the user includingpower-on, battery level, untransmitted digital ink, network connectionon-line, fault or error with an action, detection of an “active area”flag, detection of an “embedded data” flag, further data sampling torequired to acquire embedded data, acquisition of embedded datacompleted etc.

A vibration motor 446 is used to haptically convey information to theuser for important verification functions during transactions. Thissystem is used for important interactive indications that might bemissed due to inattention to the LED indicators 444 or high levels ofambient light. The haptic system indicates to the user when:

-   -   The pen wakes from standby mode    -   There is an error with an action    -   To acknowledge a transaction

5.4 Pen Optics

The pen incorporates a fixed-focus narrowband infrared imaging system.It utilizes a camera with a short exposure time, small aperture, andbright synchronised illumination to capture sharp images unaffected bydefocus blur or motion blur.

TABLE 16 Optical Specifications Magnification ^(~)0.225 Focal length of6.0 mm lens Viewing distance 30.5 mm Total track length 41.0 mm Aperturediameter 0.8 mm Depth of field 6.5 mm Exposure time 200 us Wavelength810 nm Image sensor size 140 × 140 pixels Pixel size 10 um Pitch range^(~)15 to. 45 deg Roll range ^(~)30 to. 30 deg Yaw range 0 to 360 degMinimum sampling 2.25 pixels per rate macrodot Maximum pen 0.5 m/svelocity ¹Allowing 70 micron blur radius ²Illumination and filter³Pitch, roll and yaw are relative to the axis of the pen

Cross sections showing the pen optics are provided in FIGS. 29A and 29B.An image of the Netpage tags printed on a surface 548 adjacent to thenib 418 is focused by a lens 488 onto the active region of an imagesensor 490. A small aperture 494 ensures the available depth of fieldaccommodates the required pitch and roll ranges of the pen 400.

First and second LEDs 434 and 436 brightly illuminate the surface 549within the field of view 430. The spectral emission peak of the LEDs ismatched to the spectral absorption peak of the infrared ink used toprint Netpage tags to maximise contrast in captured images of tags. Thebrightness of the LEDs is matched to the small aperture size and shortexposure time required to minimise defocus and motion blur.

A longpass IR filter 432 suppresses the response of the image sensor 490to any coloured graphics or text spatially coincident with imaged tagsand any ambient illumination below the cut-off wavelength of the filter432. The transmission of the filter 432 is matched to the spectralabsorption peak of the infrared ink to maximise contrast in capturedimages of tags. The filter also acts as a robust physical window,preventing contaminants from entering the optical assembly 470.

5.5 Pen Imaging System

A ray trace of the optic path is shown in FIG. 30. The image sensor 490is a CMOS image sensor with an active region of 140 pixels squared. Eachpixel is 10 μm squared, with a fill factor of 93%. Turning to FIG. 31,the lens 488 is shown in detail. The dimensions are:

-   -   D=3 mm    -   R1=3.593 mm    -   R2=15.0 mm    -   X=0.8246 mm    -   Y=1.0 mm    -   Z=0.25 mm

This gives a focal length of 6.15 mm and transfers the image from theobject plane (tagged surface 548) to the image plane (image sensor 490)with the correct sampling frequency to successfully decode all imagesover the specified pitch, roll and yaw ranges. The lens 488 is biconvex,with the most curved surface facing the image sensor. The minimumimaging field of view 430 required to guarantee acquisition ofsufficient tag data with each interaction is dependent on the specificcoding pattern. The required field of view for the coding patterns ofthe present invention is described in Section 4.10.

The required paraxial magnification of the optical system is defined bythe minimum spatial sampling frequency of 2.25 pixels per macrodot forthe fully specified tilt range of the pen 400, for the image sensor 490of 10 μm pixels. Typically, the imaging system employs a paraxialmagnification of 0.225, the ratio of the diameter of the inverted imageat the image sensor to the diameter of the field of view at the objectplane, on an image sensor 490 of minimum 128×128 pixels. The imagesensor 490 however is 140×140 pixels, in order to accommodatemanufacturing tolerances. This allows up to ±120 μm (12 pixels in eachdirection in the plane of the image sensor) of misalignment between theoptical axis and the image sensor axis without losing any of theinformation in the field of view.

The lens 488 is made from Poly-methyl-methacrylate (PMMA), typicallyused for injection moulded optical components. PMMA is scratchresistant, and has a refractive index of 1.49, with 90% transmission at810 nm. The lens is biconvex to assist moulding precision and features amounting surface to precisely mate the lens with the optical barrelmolding 492.

A 0.8 mm diameter aperture 494 is used to provide the depth of fieldrequirements of the design.

The specified tilt range of the pen is 15.0 to 45.0 degree pitch, with aroll range of 30.0 to 30.0 degrees. Tilting the pen through itsspecified range moves the tilted object plane up to 6.3 mm away from thefocal plane. The specified aperture thus provides a corresponding depthof field of 6.5 mm, with an acceptable blur radius at the image sensorof 16 μm.

Due to the geometry of the pen design, the pen operates correctly over apitch range of 33.0 to 45.0 degrees.

Referring to FIG. 32, the optical axis 550 is pitched 0.8 degrees awayfrom the nib axis 552. The optical axis and the nib axis converge towardthe paper surface 548. With the nib axis 552 perpendicular to the paper,the distance A between the edge of the field of view 430 closest to thenib axis and the nib axis itself is 1.2 mm.

The longpass IR filter 432 is made of CR-39, a lightweight thermosetplastic heavily resistant to abrasion and chemicals such as acetone.Because of these properties, the filter also serves as a window. Thefilter is 1.5 mm thick, with a refractive index of 1.50. Each filter maybe easily cut from a large sheet using a CO₂ laser cutter.

5.6 Electronics Design

TABLE 17 Electrical Specifications Processor ARM7 (Atmel AT91FR40162)running at 80 MHz with 256 kB SRAM and 2 MB flash memory Digital inkstorage 5 hours of writing capacity Bluetooth 1.2 Compliance USBCompliance 1.1 Battery standby 12 hours (cap off), >4 weeks (cap on)time Battery writing 4 hours of cursive writing (81% pen down, timeassuming easy offload of digital ink) Battery charging 2 hours timeBattery Life Typically 300 charging cycles or 2 years (whichever occursfirst) to 80% of initial capacity. Battery ~340 mAh at 3.7 V,Lithium-ion Polymer Capacity/Type (LiPo)

FIG. 33 is a block diagram of the pen electronics. The electronicsdesign for the pen is based around five main sections. These are:

-   -   the main ARM7 microprocessor 574,    -   the image sensor and image processor 576,    -   the Bluetooth communications module 578,    -   the power management unit IC (PMU) 580 and    -   the force sensor microprocessor 582.

5.6.1 Microprocessor

The pen uses an Atmel AT91FR40162 microprocessor (see Atmel, AT91 ARMThumb Microcontrollers—AT91FR40162 Preliminary,http://www.keil.com/dd/docs/datashts/atmel/at91fr40162.pdf) running at80 MHz. The AT91FR40162 incorporates an ARM7 microprocessor, 256 kBytesof on-chip single wait state SRAM and 2 MBytes of external flash memoryin a stack chip package.

This microprocessor 574 forms the core of the pen 400. Its dutiesinclude:

-   -   setting up the Jupiter image sensor 584,    -   decoding images of Netpage coding pattern (see Section 4.11),        with assistance from the image processing features of the image        sensor 584, for inclusion in the digital ink stream along with        force sensor data received from the force sensor microprocessor        582,    -   setting up the power management IC (PMU) 580,    -   compressing and sending digital ink via the Bluetooth        communications module 578, and    -   programming the force sensor microprocessor 582.

The ARM7 microprocessor 574 runs from an 80 MHz oscillator. Itcommunicates with the Jupiter image sensor 576 using a UniversalSynchronous Receiver Transmitter (USRT) 586 with a 40 MHz clock. TheARM7 574 communicates with the Bluetooth module 578 using a UniversalAsynchronous Receiver Transmitter (UART) 588 running at 115.2 kbaud.Communications to the PMU 580 and the Force Sensor microprocessor (FSP)582 are performed using a Low Speed Serial bus (LSS) 590. The LSS isimplemented in software and uses two of the microprocessor's generalpurpose IOs.

The ARM7 microprocessor 574 is programmed via its JTAG port.

5.6.2 Image Sensor

The ‘Jupiter’ Image Sensor 584 (see US Publication No. 2005/0024510, thecontents of which are incorporated herein by reference) contains amonochrome sensor array, an analogue to digital converter (ADC), a framestore buffer, a simple image processor and a phase lock loop (PLL). Inthe pen, Jupiter uses the USRT's clock line and its internal PLL togenerate all its clocking requirements. Images captured by the sensorarray are stored in the frame store buffer. These images are decoded bythe ARM7 microprocessor 574 with help from the ‘Callisto’ imageprocessor contained in Jupiter. The Callisto image processor performs,inter alia, low-pass filtering of captured images (see Section 4.11 andUS Publication No. 2005/0024510) before macrodot sampling and decodingby the microprocessor 574.

Jupiter controls the strobing of two infrared LEDs 434 and 436 at thesame time as its image array is exposed. One or other of these twoinfrared LEDs may be turned off while the image array is exposed toprevent specular reflection off the paper that can occur at certainangles.

5.6.3 Bluetooth Communications Module

The pen uses a CSR BlueCore4-External device (see CSR,BlueCore4-External Data Sheet rev c, 6 Sep. 2004) as the Bluetoothcontroller 578. It requires an external 8 Mbit flash memory device 594to hold its program code. The BlueCore4 meets the Bluetooth v1.2specification and is compliant to v0.9 of the Enhanced Data Rate (EDR)specification which allows communication at up to 3 Mbps.

A 2.45 GHz chip antenna 486 is used on the pen for the Bluetoothcommunications.

The BlueCore4 is capable of forming a UART to USB bridge. This is usedto allow USB communications via data/power socket 458 at the top of thepen 456.

Alternatives to Bluetooth include wireless LAN and PAN standards such asIEEE 802.11 (Wi-Fi) (see IEEE, 802.11 Wireless Local Area Networks,http://grouper.ieee.org/groups/802/11/index.html), IEEE 802.15 (seeIEEE, 802.15 Working Group for WPAN,http://grouper.ieee.org/groups/802/15/index.html), ZigBee (see ZigBeeAlliance, http://www.zigbee.org), and WirelessUSB Cypress (seeWirelessUSB LR 2.4-GHz DSSS Radio SoC,http://www.cypress.com/cfuploads/img/products/cywusb6935.pdf), as wellas mobile standards such as GSM (see GSM Association,http://www.gsmworld.com/index.shtml), GPRS/EDGE, GPRS Platform,http://www.gsmworld.com/technology/gprs/index.shtml), CDMA (see CDMADevelopment Group, http://www.cdg.org/, and Qualcomm,http://www.qualcomm.com), and UMTS (see 3rd Generation PartnershipProject (3GPP), http://www.3gpp.org).

5.6.4 Power Management Chip

The pen uses an Austria Microsystems AS3603 PMU 580 (see AustriaMicrosystems, AS3603 Multi-Standard Power Management Unit Data Sheetv2.0). The PMU is used for battery management, voltage generation, powerup reset generation and driving indicator LEDs and the vibrator motor.

The PMU 580 communicates with the ARM7 microprocessor 574 via the LSSbus 590.

5.6.5 Force Sensor Subsystem

The force sensor subsystem comprises a custom Hokuriku force sensor 500(based on Hokuriku, HFD-500 Force Sensor,http://www.hdk.co.jp/pdf/eng/e1381AA.pdf), an amplifier and low passfilter 600 implemented using op-amps and a force sensor microprocessor582.

The pen uses a Silicon Laboratories C8051F330 as the force sensormicroprocessor 582 (see Silicon Laboratories, C8051F330/1 MCU DataSheet, rev 1.1). The C8051F330 is an 8051 microprocessor with on chipflash memory, 10 bit ADC and 10 bit DAC. It contains an internal 24.5MHz oscillator and also uses an external 32.768 kHz tuning fork.

The Hokuriku force sensor 500 is a silicon piezoresistive bridge sensor.An op-amp stage 600 amplifies and low pass (anti-alias) filters theforce sensor output. This signal is then sampled by the force sensormicroprocessor 582 at 5 kHz.

Alternatives to piezoresistive force sensing include capacitive andinductive force sensing (see Wacom, “Variable capacity condenser andpointer”, US Patent Application 20010038384, filed 8 Nov. 2001, andWacom, Technology, http://www.wacom-components.com/english/tech.asp).

The force sensor microprocessor 582 performs further (digital) filteringof the force signal and produces the force sensor values for the digitalink stream. A frame sync signal from the Jupiter image sensor 576 isused to trigger the generation of each force sample for the digital inkstream. The temperature is measured via the force sensormicroprocessor's 582 on chip temperature sensor and this is used tocompensate for the temperature dependence of the force sensor andamplifier. The offset of the force signal is dynamically controlled byinput of the microprocessor's DAC output into the amplifier stage 600.

The force sensor microprocessor 582 communicates with the ARM7microprocessor 574 via the LSS bus 590. There are two separate interruptlines from the force sensor microprocessor 582 to the ARM7microprocessor 574. One is used to indicate that a force sensor sampleis ready for reading and the other to indicate that a pen down/up eventhas occurred.

The force sensor microprocessor flash memory is programmed in-circuit bythe ARM7 microprocessor 574.

The force sensor microprocessor 582 also provides the real time clockfunctionality for the pen 400. The RTC function is performed in one ofthe microprocessor's counter timers and runs from the external 32.768kHz tuning fork. As a result, the force sensor microprocessor needs toremain on when the cap 472 is on and the ARM7 574 is powered down. Hencethe force sensor microprocessor 582 uses a low power LDO separate fromthe PMU 580 as its power source. The real time clock functionalityincludes an interrupt which can be programmed to power up the ARM7 574.

The cap switch 602 is monitored by the force sensor microprocessor 582.When the cap assembly 472 is taken off (or there is a real time clockinterrupt), the force sensor microprocessor 582 starts up the ARM7 572by initiating a power on and reset cycle in the PMU 580.

5.7 Pen Software

The Netpage pen software comprises that software running onmicroprocessors in the Netpage pen 400 and Netpage pod.

The pen contains a number of microprocessors, as detailed in Section5.6. The Netpage pen software includes software running on the AtmelARM7 CPU 574 (hereafter CPU), the Force Sensor microprocessor 582, andalso software running in the VM on the CSR BlueCore Bluetooth module 578(hereafter pen BlueCore). Each of these processors has an associatedflash memory which stores the processor specific software, together withsettings and other persistent data. The pen BlueCore 578 also runsfirmware supplied by the module manufacturer, and this firmware is notconsidered a part of the Netpage pen software.

The pod contains a CSR BlueCore Bluetooth module (hereafter podBlueCore). The Netpage pen software also includes software running inthe VM on the pod BlueCore.

As the Netpage pen 400 traverses a Netpage tagged surface 548, a streamof correlated position and force samples are produced. This stream isreferred to as DInk. Note that DInk may include samples with zero force(so called “Hover DInk”) produced when the Netpage pen is in proximityto, but not marking, a Netpage tagged surface.

The CPU component of the Netpage pen software is responsible for DInkcapture, tag image processing and decoding (in conjunction with theJupiter image sensor 576), storage and offload management, hostcommunications, user feedback and software upgrade. It includes anoperating system (RTOS) and relevant hardware drivers. In addition, itprovides a manufacturing and maintenance mode for calibration,configuration or detailed (non-field) fault diagnosis. The Force Sensormicroprocessor 582 component of the Netpage pen software is responsiblefor filtering and preparing force samples for the main CPU. The penBlueCore VM software is responsible for bridging the CPU UART 588interface to USB when the pen is operating in tethered mode. The penBlueCore VM software is not used when the pen is operating in Bluetoothmode.

The pod BlueCore VM software is responsible for sensing when the pod ischarging a pen 400, controlling the pod LEDs appropriately, andcommunicating with the host PC via USB.

For a detailed description of the software modules, reference is made toUS Publication No. 2006/0028459, the contents of which are hereinincorporated by reference.

The present invention has been described with reference to a preferredembodiment and number of specific alternative embodiments. However, itwill be appreciated by those skilled in the relevant fields that anumber of other embodiments, differing from those specificallydescribed, will also fall within the spirit and scope of the presentinvention. Accordingly, it will be understood that the invention is notintended to be limited to the specific embodiments described in thepresent specification, including documents incorporated bycross-reference as appropriate. The scope of the invention is onlylimited by the attached claims.

1. A substrate having a coding pattern disposed on a surface thereof,said coding pattern comprising: a plurality of contiguous square tags oflength l, each tag comprising x-coordinate data and y-coordinate data;and a plurality of data elements contained in each tag, saidx-coordinate data being represented by a respective set of data elementsand said y-coordinate data being represented by a respective set of dataelements, wherein: all said x-coordinate data is represented in a columnof said tag parallel with a y-axis; all said y-coordinate data isrepresented in a row of said tag parallel with an x-axis; and saidcolumn and said row each have a width v, such that any square portion ofsaid coding pattern having a length (l+v) is guaranteed to contain saidx-coordinate data and said y-coordinate data for a tag irrespective ofwhether a whole tag is contained in said portion.
 2. The substrate ofclaim 1, wherein said coding pattern comprises: a plurality of targetelements defining a target grid, said target grid comprising a pluralityof cells, wherein neighboring cells share target elements and whereineach tag is defined by a plurality of contiguous cells.
 3. The substrateof claim 2, wherein each tag comprises M² contiguous square cells,wherein M is an integer having a value of at least
 1. 4. The substrateof claim 1, wherein said data elements are macrodots.
 5. The substrateof claim 1, wherein v=ts, wherein: s is defined as a spacing betweenadjacent macrodots; t is an integer value of 2 or more.
 6. The substrateof claim 4, wherein said macrodots encode data values by pulse positionmodulation (PPM).
 7. The substrate of claim 6, wherein a portion of datais represented by m macrodots, each of said macrodots occupying arespective position from a plurality of predetermined possible positionsp within said cell, the respective positions of said macrodotsrepresenting one of a plurality of possible data values.
 8. Thesubstrate of claim 7, wherein m is an integer of 1 or more, and p>m. 9.The substrate of claim 7, wherein said portion of data is a Reed-Solomonsymbol.
 10. The substrate of claim 9, wherein each cell defines a symbolgroup, each symbol group comprising a plurality of said Reed-Solomonsymbols.
 11. The substrate of claim 9, wherein said x-coordinate data isencoded as an x-coordinate codeword comprised of a respective set ofsaid X-Reed-Solomon symbols, and said y-coordinate data is encoded as ay-coordinate codeword comprised of a respective set of saidY-Reed-Solomon symbols.
 12. The substrate of claim 11, wherein saidX-Reed-Solomon symbols are configured and oriented in said column so asto have said width v, and wherein said Y-Reed-Solomon symbols areconfigured and oriented in said row so as to have said width v.
 13. Thesubstrate of claim 10, wherein each tag comprises a plurality of commoncodewords, each common codeword being comprised of a respective set ofsaid Reed-Solomon symbols, wherein said plurality of common codewordsare defined as codewords common to a plurality of contiguous tags. 14.The substrate of claim 13, wherein each symbol group comprises afragment of at least one of said common codewords, and contiguous symbolgroups are arranged such that any tag-sized portion of said codingpattern is guaranteed to contain said plurality of common codewordsirrespective of whether a whole tag is contained in said portion. 15.The substrate of claim 13, wherein said one or more of said commoncodewords encode region identity data uniquely identifying a region ofsaid surface.
 16. The substrate of claim 15, wherein said regionidentity data uniquely identifies said substrate.
 17. The substrate ofclaim 2, wherein each cell comprises a registration symbol encoded by arespective set of said data elements, said registration symbolidentifying one or more of: a translation of said cell relative to a tagcontaining said cell; an orientation of a layout of tag data withrespect to said target grid; a number of cells in each tag; a flagassociated with said tag.
 18. The substrate of claim 17, wherein eachcell comprises first and second registration symbols, said firstregistration symbol identifying a first orthogonal translation of saidcell, said second registration symbol identifying a second orthogonaltranslation of said cell.
 19. The substrate of claim 18, wherein saidfirst registration symbol identifies a first direction component of saidorientation, and said second registration symbol identifies a seconddirection component of said orientation, such that said first and secondorthogonal registration symbols together identify said orientation viasaid first and second direction components.
 20. The substrate of claim2, wherein said target elements are target dots and said data elementsare macrodots, and each target dot has a diameter of at least twice thatof each macrodot.